Making use of multiple processor cores in Ruby

Most today's computers have more than one processor core, and it's a pity not to make use of that fact, especially when using a language as slow as Ruby. Unfortunately, ruby threads all execute on the same core as the process itself, so no luck there (I think jRuby works better there, but often it's not a choice). But sometimes we can use processes instead of threads. In the following example we're doing just that, forking several processes to execute a task that's suitable for parallel processing.

#!/usr/bin/ruby

# number of simultaneous processes
sim = 4
# array of elements to process
a = (1..20).to_a

# function that processes the data
def do_the_do i
  puts "starting #{i}"
  # do something, for example, sleep between 5 and 10 seconds
  sleep(rand(5)+5)
  puts "done #{i}"
end

# starting first N processes
sim.times do
  i = a.pop
  Process.fork {do_the_do(i)}
end
# start one by one as the previous finish
a.each do |i|
  Process.wait(0)
  Process.fork {do_the_do(i)}
end
# wait for all to finish
Process.waitall
puts "done all"

The code is explained in the comments, should be pretty straightforward.


autor jablan | 14.01.09.

Komentari

Are replies also supposed to be in English? Ok.

One of the more elegant ways to use multithreading in programs is to have an "executor service" class which is given a (bounded) thread pool. The service waits on an input queue for work items, and as items arrive it block-assigns work item to the next free thread. One can vary the type of the thread pool (e.g. bounded, variable-thread, unbounded) to fine tune the executor performance. See for instance http://www.docjar.com/docs/api/java/util/concurrent/ExecutorService.html Ok, it's in Java, but the principles should still hold in other languages.

autor / author Филип Милетић | 15.01.09.

Sorry for English, I thought that these ruby related posts would be more of interest to non-serbian speaking people...

So, if I understood well, this should be similar to what you're talking about: http://snippets.dzone.com/posts/show/3276 (along with the arguable elegance which Ruby brings)? Thread pool (or, in my case, process pool) concept is definitely way to go when implementing things like this, the code I wrote would be simplified version of it, considering the fact that I didn't have the need for listening the queue: all the work items are known in advance.

autor / author jablan | 15.01.09.

Yes, that's it. The thread/process pool has the advantage that it packs the threading issues (synchronization and checking for processes that terminate) in a black box and you don't need to worry about them once it's in place; thus separating your business logic from process management logic. As the two are unrelated for most of the time, it makes the code easier to understand.

autor / author Филиип Милетић | 15.01.09.

Dodajte svoj komentar:



Obavesti me kad neko odgovori na ovaj zapis? / Notify me when someone responds to this entry?

O sajtu
Autori
FAQ
Linkovi

Kategorije

Lično
Opšte
Pretraživači
Razvoj
Softver
Veb

Pretraga sajta

Arhiva

po datumu
po kategoriji

RSS 0.91

Powered by
pMachine