[ruby-core:114804] [Ruby master Bug#19890] File#realine(chomp: true) slower/more allocations than readline.chomp!

Issue #19890 has been updated by tenderlovemaking (Aaron Patterson).





This is an implementation detail, but IO#readline is [implemented as a C fu=
nction](https://github.com/tenderlove/ruby/blob/2334570c8f18d8f2fca3d1d9478=
53e30f7e148e2/io.c#L4371-L4373), and currently there is no way to pass keyw=
ord args to a C function without allocating a hash.  I re-implemeted IO#rea=
dline in Ruby and it does eliminate the allocation overhead (I've not teste=
d performance).



I sent the patch [here](https://github.com/ruby/ruby/pull/8473).



More implementation details, but the challenge with this patch is that `IO#=
readline` sets `$_` to the last read line, but it does that only in the cal=
ler's frame.  Take the following program as an example:



```ruby

class Foo

  def call

    File.open(__FILE__) do |f|

      read f

      p __method__ =3D> $_

    end

  end



  def read f

    f.readline

    p __method__ =3D> $_

  end

end



Foo.new.call

```



If you run this, the output is:



```

$ ruby -v test.rb

ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]

{:read=3D>"class Foo\n"}

{:call=3D>nil}

```



So `$_` looks like a global, but it's not really.  `$_` is set in the `read=
` method, but not set in the `call` method.



The callee (`IO#readline`) is manipulating values _in the caller's_ environ=
ment!  The last line value is written via [`rb_lastline_set`](https://githu=
b.com/tenderlove/ruby/blob/501aeb3432fd1ed4b4827841b287bf7a44a3c0a4/vm.c#L1=
743-L1747).  This function eventually [walks up the call stack, looking for=
 the most recent Ruby call frame](https://github.com/tenderlove/ruby/blob/5=
01aeb3432fd1ed4b4827841b287bf7a44a3c0a4/vm.c#L1679-L1691) and sets `$_` in =
that frame.  Since `IO#readline` was implemented in C, "the most recent Rub=
y call frame" is the user code (in my example `Foo#read`).



However, moving `IO#readline` to Ruby means that "the most recent Ruby call=
 frame" is `IO#readline` itself.  Of course, setting `$_` in `IO#readline` =
is of no use to anyone, so in my PR I [added a function that that lets us s=
et special variables in other frames](https://github.com/ruby/ruby/pull/847=
3/files#diff-2af2e7f2e1c28da5e9d99ad117cba1c4dabd8b0bc3081da88e414c55c6aa95=
49R1749-R1750).  I'm not particularly thrilled with this because it's coupl=
ing the implementation of `IO#readline` with its stack depth.  That said, i=
t supports `$_` and fixes this issue.



I think it would be cool if we could push a special frame for methods like =
`IO#readline` so that we know where user code is, but I feel a solution lik=
e that is beyond the scope of this ticket.



----------------------------------------

Bug #19890: File#realine(chomp: true) slower/more allocations than readline=
.chomp!

https://bugs.ruby-lang.org/issues/19890#change-104648



* Author: segiddins (Samuel Giddins)

* Status: Open

* Priority: Normal

* ruby -v: 3.2.2

* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

----------------------------------------

On ruby 3.2.2 running the following script:



``` ruby

#!/usr/bin/env ruby



require 'rubygems'

require 'bundler/inline'



puts RUBY_VERSION



gemfile do

  source "https://rubygems.org"

  gem "benchmark-ipsa"

end



Benchmark.ipsa do |x|

  x.report("f.readline(chomp: true)") do

    File.open("/usr/share/dict/words") do |f|

      f.readline(chomp: true) until f.eof?

    end

  end

 =20

  x.report("f.readline.chomp!") do

    File.open("/usr/share/dict/words") do |f|

      until f.eof?

        s =3D f.readline

        s.chomp!

        s

      end

    end

  end

 =20

  x.report("f.readline.chomp") do

    File.open("/usr/share/dict/words") do |f|

      until f.eof?

        f.readline.chomp

      end

    end

  end

 =20

  x.compare!

end

```



I get the following (surprising) result:



```

3.2.2

Allocations -------------------------------------

	f.readline(chomp: true)

                      707931/1  alloc/ret       50/1  strings/ret

   f.readline.chomp!  235979/1  alloc/ret       50/1  strings/ret

    f.readline.chomp  471955/1  alloc/ret       50/1  strings/ret

Warming up --------------------------------------

f.readline(chomp: true)

                         1.000  i/100ms

   f.readline.chomp!     2.000  i/100ms

    f.readline.chomp     2.000  i/100ms

Calculating -------------------------------------

f.readline(chomp: true)

                         16.165  (=B1 6.2%) i/s -     81.000=20

   f.readline.chomp!     25.246  (=B1 7.9%) i/s -    126.000=20

    f.readline.chomp     20.997  (=B1 9.5%) i/s -    106.000=20



Comparison:

   f.readline.chomp!:       25.2 i/s

    f.readline.chomp:       21.0 i/s - 1.20x slower

f.readline(chomp: true):       16.2 i/s - 1.56x slower

```



I would expect `File#readline(chomp: true)` to be comparable to `s =3D f.re=
adline; s.chomp!; s` at a bare minimum, but it is slower and has more alloc=
ations even than `readline.chomp`









--=20

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-c=
ore.ml.ruby-lang.org/