Re: Threading in ruby

On Tue, Jan 4, 2011 at 8:41 AM, Vishnu I. <[e-mail]> wrote:

> =A0 =A0I was recently trying to solve a pet problem of mine where threadi=
ng
> seems to be the most natural way to solve a problem. I needed a thread
> that would scan a fixed list of tuples each of which contained a single
> filenames to sha1 them and add the sha1 to each tuple and a second
> thread that would run through this list and contact this service on the
> internet to get metadata on these files. (the service exposes a udp api
> that allows a given ip to use only a fixed local port and has strict
> rate limits, so I don't want to request metadata for files
> concurrently). The thing is I have almost never worked worked with
> threading since I've mostly built web apps.

This is the point where I would immediately switch to using a queue.

> =A0So I decided to maintain a list of tuples and a last updated index tha=
t
> would be visible to both threads.
>
> So something like this
> https://gist.github.com/764495#file_simple_multithreaded_example .
> However, I understand that this may not work. Because there is no
> guarantee of the order in which the two operations are performed and
> these might be reordered. But this understanding comes from reading
> about this in other languages. Is this also true in Ruby?

Yes.  Without synchronization of some sort all bets are off and there
are no guarantees about the order in which code in concurrent threads
is executed.

> I understand that if I were doing this in .net or java I could just make
> index volatile which is a directive to the compiler and runtime that
> operations surrounding it should not be reordered. Am i correct in
> understanding that there is no such concept in ruby? So I cannot do this
> locklessly.

Exactly.

> So I decided to lock on a mutex and I came up with this.
> https://gist.github.com/764495#file_simple_multithreaded_example_with_loc=
ks
> now in the scanner thread I have udpated the list inside the sychronized
> block and then updated index,

As far as I can see you forgot to increment local_index after #update_metad=
ata.

> but is it safe to move the list updation
> outside? (I assume no since regular statements and sych blocks can get
> reordered).

There is no list update.  You only update Hash instances contained in
the Array.  With your logic it should be safe to only synchronize on
the index access because you modify only those entries in the Array
with index > current index.

> Also I have sychronized in the updater thread because I dont
> know if statements inside sychronized blocks can be re-ordered. Is that
> right?

Not sure what you mean here.  You properly synchronize access to index
which is exactly what you need to do to make the code safe.

> p.s. After I considered this solution, I came to know about the Queue
> class=A0in thread, so I know that that is a more elegant solution. But I
> would still like to know the solution.

https://gist.github.com/764540

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/