From: nobu@... Date: 2017-01-29T01:02:18+00:00 Subject: [ruby-core:79298] [Ruby trunk Bug#13085][Assigned] io.c io_fwrite creates garbage Issue #13085 has been updated by Nobuyoshi Nakada. Status changed from Open to Assigned Assignee set to Eric Wong Seems nice, let's try it. ---------------------------------------- Bug #13085: io.c io_fwrite creates garbage https://bugs.ruby-lang.org/issues/13085#change-62711 * Author: Eric Wong * Status: Assigned * Priority: Normal * Assignee: Eric Wong * Target version: * ruby -v: * Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- Relying on rb_str_new_frozen for unconverted strings does not save memory because copy-on-write is always triggered in read-write I/O loops were subsequent IO#read calls will clobber the given write buffer. ```ruby buf = ''.b while input.read(16384, buf) output.write(buf) end ``` This generates a lot of garbage starting with Ruby 2.2 (r44471). For my use case, even `IO.copy_stream` generates garbage, since I wrap "write" to do Digest calculation in a single pass. I tried using rb_str_replace and reusing the string as a hidden `(klass == 0)` thread-local, but `rb_str_replace` attempts CoW optimization by creating new frozen objects, too: https://80x24.org/spew/20161229004417.12304-1-e@80x24.org/raw So, I'm not sure what to do, temporal locking seems wrong for writing strings (I guess it's for reading?). I get `test_threaded_flush` failures with the following: https://80x24.org/spew/20161229005701.9712-1-e@80x24.org/raw `IO#syswrite` has the same problem with garbage. I can use `IO#write_nonblock` on fast filesystems while holding GVL, I guess... ---Files-------------------------------- 0001-io.c-io_fwrite-temporarily-freeze-string-when-writin.patch (2.6 KB) 0001-v2-io.c-io_fwrite-copy-to-hidden-buffer-when-writing.patch (2.9 KB) 0001-basicsocket-Linux-workaround-for-excess-garbage-on-w.patch (5.54 KB) 0001-io.c-recycle-garbage-on-write.patch (6.33 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: