From: Eric Wong via ruby-core <ruby-core@...>
Date: 2023-05-20T10:25:20+00:00
Subject: [ruby-core:113543] Re: [Ruby master Feature#19642] Remove vectored read/write from `io.c`.

"ioquatix (Samuel Williams) via ruby-core" <ruby-core@ml.ruby-lang.org> wrote:
> ```
> irb(main):001:0> $stderr.sync
> => true
> irb(main):002:0> $stdout.sync
> => true
> ```
> 
> It looks like `$stdout` and `$stderr` are both buffered internally.

IO#sync==true means unbuffered.

> I think Ruby should guarantee buffered writes will be atomic up to PIPE_BUF.

PIPE_BUF is only relevant for pipes/FIFOs (not regular files, sockets, etc..).

> Unbuffered writes should have no atomicity guarantees.

Each write/writev syscall to regular files is atomic with O_APPEND.
It's how multi-process servers can write to the same log file
without interleaving lines.

The limit is not PIPE_BUF, but (IIRC) roughly INT_MAX/SSIZE_MAX;
It may be FS-dependent, but it's large enough to not matter on
reasonable local FSes.


Side notes:


Also, any writev benchmarks you'd do should account for the
common pattern of giant bodies prefixed with a small header.
(e.g. HTTP/1.1 chunking, tnetstrings(*), etc).

The 128 bytes you use in the benchmark is tiny and unrealistic.
writev has high overhead with many small chunks (each iovec is
16 bytes), and most I/O size is far larger than 128 bytes.

(*) https://tnetstrings.info/

I understand that writev is slow for the kernel, but (alloc +
memcpy + GC/free) w/ giant strings is slow in userspace, too.

Stuff like:

	io.write("#{buf.bytesize.to_s(16)}\r\n", buf, -"\r\n")

When buf is a gigantic string (multiple KB or MB) is where I
expect writev to avoid copies and garbage.



Adding a buffer_offset (tentative name) arg to write_nonblock
would make retrying non-blocking I/O much easier:

  n = 0
  tot = ary.map(&:bytesize).sum
  while n < tot
    case w = io.write_nonblock(*ary, buffer_offset: n, exception: false)
    when :wait_readable, :wait_writable
      break # or schedule or whatever
    else
      n += w
    end
  end

Perl's syswrite/read/sysread ops have the OFFSET arg for decades.
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/