[ruby-core:78921] [Ruby trunk Bug#13085] io.c io_fwrite creates garbage

Issue #13085 has been updated by Eric Wong.

File 0001-io.c-io_fwrite-temporarily-freeze-string-when-writin.patch added

Proposed patch to temporarily freeze string while copying
~~~
    io.c (io_fwrite): temporarily freeze string when writing
    
    This avoids garbage from IO#write for [Bug #13085].
    Memory usage from benchmark/bm_io_copy_stream_write.rb
    is reduced greatly:
    
      target 0: a (ruby 2.5.0dev (2016-12-30 trunk 57236) [x86_64-linux])
      target 1: b (ruby 2.5.0dev (2016-12-30) [x86_64-linux])
    
      Memory usage (last size) (B)
      name  a       b
      io_copy_stream_write  82235392.000    6651904.000
    
      Memory consuming ratio (size) with the result of `a' (greater is better)
      name  b
      io_copy_stream_write  12.363
    
    There is also a speedup in execution time:
    
      Execution time (sec)
      name  a       b
      io_copy_stream_write  0.380   0.143
    
      Speedup ratio: compare with the result of `a' (greater is better)
      name  b
      io_copy_stream_write  2.651
    
    Caveat, there is one potential race condition:
    
    If another thread calls String#freeze on the string we are
    currently writing; we will blindly unfreeze it during
    fwrite_unfreeze from ensure.  However, I do not expect this to
    be a real-world case.
    
    Ideally, Ruby should have a way of detecting threads which
    are not visible to other threads.
~~~


----------------------------------------
Bug #13085: io.c io_fwrite creates garbage
https://bugs.ruby-lang.org/issues/13085#change-62335

* Author: Eric Wong
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
Relying on rb_str_new_frozen for unconverted strings does not
save memory because copy-on-write is always triggered in
read-write I/O loops were subsequent IO#read calls will
clobber the given write buffer.

  buf = ''.b
  while input.read(16384, buf)
    output.write(buf)
  end

This generates a lot of garbage starting with Ruby 2.2 (r44471).
For my use case, even IO.copy_stream generates garbage, since
I wrap "write" to do Digest calculation in a single pass.

I tried using rb_str_replace and reusing the string as a hidden
(klass == 0) thread-local, but rb_str_replace attempts CoW
optimization by creating new frozen objects, too:

  https://80x24.org/spew/[e-mail]/raw


So, I'm not sure what to do, temporal locking seems wrong for
writing strings (I guess it's for reading?).  I get
test_threaded_flush failures with the following:

  https://80x24.org/spew/[e-mail]/raw


IO#syswrite has the same problem with garbage.  I can use
IO#write_nonblock on fast filesystems while holding GVL,
I guess...


---Files--------------------------------
0001-io.c-io_fwrite-temporarily-freeze-string-when-writin.patch (2.6 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>