[ruby-core:79103] [Ruby trunk Bug#13085] io.c io_fwrite creates garbage

Issue #13085 has been updated by Eric Wong.

File 0001-basicsocket-Linux-workaround-for-excess-garbage-on-w.patch added

At least on Linux, this is probably the best performance we can get
with sockets.  Unfortunately, this does not affect writes to pipes
(write_nonblock is OK for me, at least), and regular files
(where we really need to release GVL)

--------8<-------
Subject: [PATCH] basicsocket: Linux workaround for excess garbage on write

Linux allows us to use the MSG_DONTWAIT flag on sockets
to do non-blocking send(2) without side effects of modifying
the underlying file flags.  This allows us to replace the
generic IO#write and avoid garbage creation on GVL release:
<https://bugs.ruby-lang.org/issues/13085>
We now only release the GVL when IO cannot proceed.

While IO#write_nonblock may be used for pipes and sockets
portably, that has the side effect of changing the file flag via
fcntl.  MSG_DONTWAIT under Linux has no side effects on the
socket.

I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.
I HATE IMPLICITLY SHARED MEMORY.


----------------------------------------
Bug #13085: io.c io_fwrite creates garbage
https://bugs.ruby-lang.org/issues/13085#change-62512

* Author: Eric Wong
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
Relying on rb_str_new_frozen for unconverted strings does not
save memory because copy-on-write is always triggered in
read-write I/O loops were subsequent IO#read calls will
clobber the given write buffer.

  ```ruby
  buf = ''.b
  while input.read(16384, buf)
    output.write(buf)
  end
  ```

This generates a lot of garbage starting with Ruby 2.2 (r44471).
For my use case, even `IO.copy_stream` generates garbage, since
I wrap "write" to do Digest calculation in a single pass.

I tried using rb_str_replace and reusing the string as a hidden
`(klass == 0)` thread-local, but `rb_str_replace` attempts CoW
optimization by creating new frozen objects, too:

  https://80x24.org/spew/[e-mail]/raw


So, I'm not sure what to do, temporal locking seems wrong for
writing strings (I guess it's for reading?).  I get
`test_threaded_flush` failures with the following:

  https://80x24.org/spew/[e-mail]/raw


`IO#syswrite` has the same problem with garbage.  I can use
`IO#write_nonblock` on fast filesystems while holding GVL,
I guess...


---Files--------------------------------
0001-io.c-io_fwrite-temporarily-freeze-string-when-writin.patch (2.6 KB)
0001-v2-io.c-io_fwrite-copy-to-hidden-buffer-when-writing.patch (2.9 KB)
0001-basicsocket-Linux-workaround-for-excess-garbage-on-w.patch (5.54 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>