[ruby-core:45898] [ruby-trunk - Feature #6615] Release GVL in zlib when calling inflate() or deflate()

Issue #6615 has been updated by drbrain (Eric Hodel).


This benchmark inflates ~43,000 100KB deflate strings (~1GB total input) using the same ruby version as above.

Code:

  require 'zlib'
  require 'benchmark'
  
  r = Random.new 0
  
  file_count = 2**30 / 100_000 # 100KB chunks in 1GB
  
  deflated = (0..file_count).map do
    input = r.bytes 100_000
    Zlib::Deflate.deflate input
  end
  
  times = Benchmark.measure do
    deflated.each do |input|
      Zlib::Inflate.inflate input
    end
  end
  
  puts times
  
Without patch:

  $ ruby20 test.rb 
    1.350000   0.040000   1.390000 (  1.378062)
  $ ruby20 test.rb 
    1.330000   0.010000   1.340000 (  1.343311)
  $ ruby20 test.rb 
    1.350000   0.020000   1.370000 (  1.363618)
  $ ruby20 test.rb 
    1.430000   0.030000   1.460000 (  1.464801)

With patch:

  $ make runruby
  ./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems ./test.rb
    1.170000   0.020000   1.190000 (  1.198120)
  $ make runruby
  ./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems ./test.rb
    1.250000   0.020000   1.270000 (  1.273507)
  $ make runruby
  ./miniruby -I./lib -I. -I.ext/common  ./tool/runruby.rb --extout=.ext  -- --disable-gems ./test.rb
    1.320000   0.010000   1.330000 (  1.333873)

So there is a slight increase in performance, probably due to use of realloc vs REALLOC_N
----------------------------------------
Feature #6615: Release GVL in zlib when calling inflate() or deflate()
https://bugs.ruby-lang.org/issues/6615#change-27498

Author: drbrain (Eric Hodel)
Status: Open
Priority: Normal
Assignee: 
Category: ext
Target version: 2.0.0


This patch switches from zstream_run from using rb_thread_schedule() to rb_thread_blocking_region().

I don't see a way to safely interrupt deflate() or inflate() so the unblocking function is empty.

This patch should allow use of output buffer sizes larger than 16KB.  I suspect 16KB was chosen to allow reasonable context-switching time for ruby 1.8 and earlier.  A larger buffer size would reduce GVL contention when processing large streams.

An alternate way to reduce GVL contention would be to move zstream_run's loop outside the GVL, but some manual allocation would be required as currently the loop uses a ruby String as the output buffer.


-- 
http://bugs.ruby-lang.org/