From: Eleanor McHugh <eleanor@...>
Date: 2008-06-30T19:45:10+09:00
Subject: Re: multi-threaded access to shared memory space

On 30 Jun 2008, at 07:58, Greg Willits wrote:
> I have a pure Ruby project (no Rails) where I would like multiple
> "tasks" (ruby processes more or less) to run in parallel (collectively
> taking advantage of multiple CPU cores) while accessing a shared  
> memory
> space of data structures.
>
> OK, that's a mouthful.
>
> - single machine, multiple cores (4 or 8)
>
> - step one: pre-load a number of arrays and hashes (could be a  
> couple GB
> worth in total) into memory
>
> - step two: launch several independent Ruby scripts to search and read
> from the data pool in order to aggregate data in new sets to be  
> written
> to text files.
>
> Ruby 1.8's threading would seem poorly suited to this. Can 1.9 run
> multiple threads each accesing the same RAM-space while using all  
> cores
> of the machine?
>
> I've looked at memcache, but it seems like it could store and retrieve
> one of my pool's arrays, but it cannot look inside that array and
> retrieve just a single row of it? It would want to return the whole
> array, yes? (not good if that array is 100MB).

If you want to stay in pure Ruby, take a look at DRb and Rinda. Even  
if not directly applicable they should give you some inspiration.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net
----
raise ArgumentError unless @reality.responds_to? :reason