0

Current I have the following code to run some ruby scripts, within a ruby script:

def run(base_directory, run_count)
  working_directory = base_directory.gsub("\n","")
  for i in 1..run_count
    system("ruby " + working_directory + i.to_s + "\\" + "main.rb " +   working_directory + i.to_s + "\\")
  end
end

However this runs the scripts in a sequence, but I need them to run in parallel. Where I have 10 scripts to run, and I want to run 5 at a time until I reach the number of scripts that needs run. Is there a simple way to accomplish this?

David Brewer
  • 1,864
  • 8
  • 25
  • 37

2 Answers2

5

Just discovered this gem parallel. You probably will have to run it like this:

results = Parallel.map(run_count.downto(1).to_a, :in_processes=>run_count){|i| system("ruby " + working_directory + i.to_s + "\\" + "main.rb " +   working_directory + i.to_s + "\\") }
  • Alright so I started down the path of using parallel because it seems like the best choice. But i've hit a road block with this stack track: https://gist.github.com/dgbrewer1989/12e657496818b69a6bb0 It it seems that it can't find fork.. but it's installed. I'm unsure if this is a gem issue, or ruby issue. – David Brewer Aug 18 '15 at 01:00
  • Have you tried to search that error? I see several results about this error after a simple googling for "fork() function is unimplemented on this machine". You have to go through them or even open in issue for that gem in their github account (they might redirect you back to stackoverflow and ask that question here). –  Aug 18 '15 at 18:12
  • Yeah I researched it after posting and it seems like it's a windows issue where the fork() command isn't supported on windows. – David Brewer Aug 18 '15 at 18:45
  • 1
    I actually wanted to ask you whether you are under Windows. I had many similar issues when I was learned to develop ROR under Windows. I have long time switched to Ubuntu since, and never used Windows again for anything (rather than very specific tasks). After all most of the servers you will be dealing with will be Ubuntu. –  Aug 18 '15 at 22:02
  • Good point. I would be okay making it in Ubuntu.. but it's going to be run in Windows down the line anyway. Don't wanna have to come back to this later and deal with the issue. – David Brewer Aug 19 '15 at 13:01
  • 2
    @DavidBrewer: You might try to install sygwin on your windows server and run linux-oriented gems with it. Sygwin was the second thing that I tried under Windows before switching to Ubuntu, and it went well for a while, but at the end sygwin also had its limits. –  Aug 20 '15 at 05:30
0

Not sure what is the proper way, but I usually just run a one-liner that executes something from the command line:

Say you have a file called create_file_with_input.rb that does whatever, like one line with: File.open("file_#{ARGV[0]}.txt", 'a').

you can do:

run_count = 5

%x(for i in `seq 0 #{run_count-1}`; do (ruby create_file_with_input.rb $i &); done)

That would create 5 files like (file_0.txt, file_1.txt, ..., file_4.txt etc.). Tweak it to fit your code from above and BOOM. You have multiple scripts running.

You could also potentially use fork, although I don't have much personal experience with it. See this question (or use The Google) for forking child processes. This way also provides the benefit of getting the PID of child processes.

Community
  • 1
  • 1
Ty.Sp
  • 7
  • 3
  • I'm a little confused by how the example works. 1: Not sure what "%x" means. 2: It's looping from i (starting at 0?) until the run_count - 1 and what's it's doing 3: Is it's running ruby then the file name. Is the $i the i from the for loop? And what's the & symbol for? Sorry for all the questions.. I just want to understand what I'm working with. – David Brewer Aug 18 '15 at 01:09
  • @DavidBrewer no problem. 1. `%x` is just another way to run a system command in ruby (basically the same as backticks). [See here](https://gist.github.com/JosephPecoraro/4069) for more info. 2. yes, `i` is looping from 0 to 4 in this case (the loop with the system `seq` cmd is inclusive, i.e. it will run 5x.), and it is the same as `$i` -- you don't have to use it but can pass it to the ruby file (as shown) if you want. 3. the & symbol just detaches the command from the current shell ( [see here](http://unix.stackexchange.com/questions/110822/using-bash-operator-with-delineator) ) – Ty.Sp Aug 18 '15 at 01:58