Issue #11625 has been updated= by Aaron Patterson.

File sha1gvl.diff added

Hi Eric,

Thanks for the feedback.=C2=A0 I've updated the patch will your suggest= ions.=C2=A0 Thank you!

sha1.h exports SHA1_Transform, so I was worried about touching that one.
----------------------------------------
Feature #11625: Unlock GVL for SHA1 calculations
https://bugs.ruby-lang.org/issues/11625#change-5= 4608

* Author: Aaron Patterson
* Status: Open
* Priority: Normal
* Assignee:
----------------------------------------
I'm trying to calculate many sha1 checksums, but the current sha1 imple= mentation doesn't unlock the GVL, so I can't do it in parallel.=C2= =A0 I've attached a patch that unlocks the GVL when calculating sha1sum= s so that I can do them in parallel.

The good point about this patch is that I can calculate sha1's in paral= lel.=C2=A0 Here is the test code I'm using:

~~~
require 'digest/sha1'
require 'thread'

Thread.abort_on_exception =3D true

THREADS =3D (ENV['THREADS'] || 1).to_i

store =3D 'x' * (ENV['SIZE'] || 1024).to_i

queue =3D Queue.new

600000.times do
=C2=A0 queue << store
end

THREADS.times { queue << nil }

ts =3D THREADS.times.map {
=C2=A0 Thread.new {
=C2=A0 =C2=A0 while work =3D queue.pop
=C2=A0 =C2=A0 =C2=A0 Digest::SHA1.hexdigest(work)
=C2=A0 =C2=A0 end
=C2=A0 }
}

ts.each(&:join)
~~~

Here is what the output looks like after I've applied the patch:

~~~
[aaron@TC ruby (trunk)]$ THREADS=3D1 SIZE=3D4096 time ./ruby test.rb
=C2=A0 =C2=A0 =C2=A0 =C2=A022.62 real=C2=A0 =C2=A0 =C2=A0 =C2=A0 21.78 user= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.66 sys
[aaron@TC ruby (trunk)]$ THREADS=3D4 SIZE=3D4096 time ./ruby test.rb
=C2=A0 =C2=A0 =C2=A0 =C2=A015.87 real=C2=A0 =C2=A0 =C2=A0 =C2=A0 34.53 user= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A08.27 sys
[aaron@TC ruby (trunk)]$
~~~

The digests that I'm calculating are for fairly large strings, so this = patch works well for me.=C2=A0 The downsides are that it seems slightly slo= wer (though I'm not sure that it's significant) with a single threa= d:

Test code:

~~~
require 'benchmark/ips'
require 'digest/sha1'

Benchmark.ips do |x|
=C2=A0 x.report('sha1') { Digest::SHA1.hexdigest('x' * 4096= ) }
end
~~~

Before my patch (higher numbers are better):

~~~
[aaron@TC ruby (trunk)]$ ./ruby shaips.rb
Calculating -------------------------------------
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sha1=C2=A0 =C2=A0 = =C2=A02.604k i/100ms
-------------------------------------------------
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sha1=C2=A0 =C2=A0 = =C2=A027.441k (=C2=B1 3.9%) i/s -=C2=A0 =C2=A0 138.012k
~~~

After my patch:

~~~
[aaron@TC ruby (trunk)]$ ./ruby shaips.rb
Calculating -------------------------------------
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sha1=C2=A0 =C2=A0 = =C2=A02.419k i/100ms
-------------------------------------------------
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sha1=C2=A0 =C2=A0 = =C2=A025.848k (=C2=B1 2.8%) i/s -=C2=A0 =C2=A0 130.626k
~~~

Other downside is that I changed the `update` method to dup strings so that= the GVL can be safely released.

This patch pays off for me because of the size of the strings I'm worki= ng with, but I'm not sure if it's fine for the general case.

---Files--------------------------------
sha1gvl.diff (3.16 KB)
sha1gvl.diff (3.79 KB)

--
https://bugs.ruby-lang.org/