From: "znz (Kazuhiro NISHIYAMA) via ruby-core" Date: 2026-11-11T09:29:45+00:00 Subject: [ruby-core:127416] [Ruby Bug#22537] `IO#gets(nil, limit)` stops right after a 0xFF byte Issue #22537 has been reported by znz (Kazuhiro NISHIYAMA). ---------------------------------------- Bug #22537: `IO#gets(nil, limit)` stops right after a 0xFF byte https://bugs.ruby-lang.org/issues/22537 * Author: znz (Kazuhiro NISHIYAMA) * Status: Open * ruby -v: ruby 4.1.0dev (2026-11-09T07:44:48Z HEAD 4f0a0a0eca) +PRISM [x86_64-linux] * Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- `IO#gets(nil, limit)` (no separator, with a limit) returns early when the data contains a `0xFF` byte: it returns the bytes up to and including the first `0xFF`, even though neither the limit nor EOF has been reached. `IO#gets(nil)` and `IO#read(limit)` return the whole data. ```ruby puts RUBY_DESCRIPTION File.binwrite("t.bin", "ab\xFFcd") File.open("t.bin", "rb") { |f| p f.gets(nil, 100) } File.open("t.bin", "rb") { |f| p f.gets(nil) } File.open("t.bin", "rb") { |f| p f.read(100) } File.open("t.bin", "rb") { |f| p f.each_line(nil, 100).to_a } File.binwrite("t.bin", "ab\xFEcd") File.open("t.bin", "rb") { |f| p f.gets(nil, 100) } ``` ``` ruby 4.1.0dev (2026-11-09T07:44:48Z HEAD 4f0a0a0eca) +PRISM [x86_64-linux] "ab\xFF" "ab\xFFcd" "ab\xFFcd" ["ab\xFF", "cd"] "ab\xFEcd" ``` Expected: `"ab\xFFcd"` for `gets(nil, 100)` and `["ab\xFFcd"]` for `each_line(nil, 100)`, because `nil` means "no separator" and the data is shorter than the limit. The output is the same on 4.0.0 and 3.0.0, and `gets(nil, 100)` also returns `"ab\xFF"` on 2.7.0, 2.0.0 and 1.9.3. Ruby 1.9.1 returns `"ab\xFFcd"`. `IO#readline(nil, 100)` and a pipe behave the same way as the file above. `StringIO#gets(nil, 100)` returns the whole string. Other bytes such as `0xFE` are not affected. It looks like the cause is the `-1` used as "no delimiter" in `rb_io_getline_0`: https://github.com/ruby/ruby/blob/v4.0.0/io.c#L4200 When `rs` is `nil`, `newline` stays `-1` and is passed to `appendline` as `delim`, and `search_delim` calls `memchr(p, delim, len)` when the minimum character length of the encoding is 1: https://github.com/ruby/ruby/blob/v4.0.0/io.c#L3882-L3887 `memchr` converts its second argument to `unsigned char`, so `-1` matches `0xFF`. I found this through the csv library: `CSV.read(path)` reads a sample with `@input.gets(nil, chunk_size)` to detect the row separator, and a file that contains `0xFF` gets a truncated sample (https://github.com/ruby/csv/issues/385). -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/