[ruby-core:108944] [Ruby master Bug#18833] Documentation for IO#gets is inaccurate (bytes versus characters)

Issue #18833 has been updated by adh1003 (Andrew Hodgkinson).


For avoidance of doubt, the behaviour of Ruby itself is (IMHO) sensible and working well. The only change needed is to alter the word "bytes" to "characters" for the `IO#gets` description of the `limit` parameter.

----------------------------------------
Bug #18833: Documentation for IO#gets is inaccurate (bytes versus characters)
https://bugs.ruby-lang.org/issues/18833#change-98039

* Author: adh1003 (Andrew Hodgkinson)
* Status: Open
* Priority: Normal
* ruby -v: N/A
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
Please see https://ruby-doc.org/core-3.1.2/IO.html#method-i-gets:

> With integer argument `limit` given, returns up to `limit+1` bytes:

In relation to https://github.com/janko/down/pull/74, I discovered that while `IO#read` ignores the stream's specified encoding if asked to read a specific number of bytes and does then do exactly that - reads the requested number of 8-bit bytes - `IO#gets` respects the encoding if given a `limit` and the **number provided is characters, not bytes**. This means that not only might more actual bytes be read from the file (advancing its file pointer accordingly) due to things like a BOM, more bytes might also be read for multi-byte encoding. Moreover, the number of bytes in the returned data can exceed the number passed to the method (because it's a number of characters, contrary to the documentation) and it won't necessarily include some bytes from the very start of the file (a UTF-8 BOM is stripped, for example). `IO#gets` *does* correctly handle a multibyte character being split at the limit of the requested read position if taken as bytes and continues reading more bytes until it has read the requested number of complete characters.

(It is in fact clearly unavoidable that it works in an encoding-aware fashion, else it would be unable to accurately interpret the `sep` parameter. Coercing everything down to a pure 8-bit byte stream and trying to dumb-match the stream that way would risk mismatching a separator byte stream within the wider file byte stream at a non-character boundary).

This is causing confusion for people implementing IO subclasses or IO-like classes and I'm sure you recognise that it is of critical importance that the distinction between bytes and characters is made accurately, especially in such a crucial low-level piece of documentation as IO.

If you wish, I can have a go at figuring out a PR for it (not really done that ouside of GitHub before, so something of a learning curve!).





-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>