From: Robert Klemme <bob.news@...>
Date: 2004-09-04T18:30:20+09:00
Subject: Re: autochomp?


"Kristof Bastiaensen" <kristof@vleeuwen.org> schrieb im Newsbeitrag
news:pan.2004.09.03.22.50.31.89211@vleeuwen.org...
> Hi,
>
> On Fri, 03 Sep 2004 18:03:35 +0200, Robert Klemme wrote:
>
> >
> > "Kristof Bastiaensen" <kristof@vleeuwen.org> schrieb im Newsbeitrag
> > news:pan.2004.09.02.20.13.44.101750@vleeuwen.org...
> >> Hi,
> >>
> >> On Thu, 02 Sep 2004 19:18:10 +0200, Robert Klemme wrote:
> >>
> >>
> >> > "Martin DeMello" <martindemello@yahoo.com> schrieb im Newsbeitrag
> >> > news:idIZc.301315$J06.4478@pd7tw2no...
> >> >> [quoted text muted]
> >> >
> >> > Not AFAIK.  You can do
> >> >
> >> > content = IO.readlines(file).each {|l| l.chomp!}
> >> >
> >> > or (more efficient)
> >> >
> >> > content = File.open(file) {|io| io.inject([]) {|ar,line| line.chomp!;
> >> > ar << line} }
> >> >
> >> >
> >> I think in this case the first is more efficient than the second. Since
> >> chomp! modifies the object inplace, no object gets created.
> >
> > ???  Please read again.  The first is less efficient because it iterates
> > through the lines of the file twice and both use inplace modification
> > (#chomp!).
> >
> > Kind regards
> >
> >     robert
>
> It would seem logical, but benchmarking shows different:
>
> #---- begin filetest.rb ---- 
> require 'benchmark'
>
> file = "testfile.txt"
> Benchmark.bm(7) do |x|
>    x.report("readlines:") do
>       IO.readlines(file).each {|l| l.chomp!}
>    end
>
>    x.report("inject:") do
>       File.open(file) {|io| io.inject([]) {
>             |ar,line| line.chomp!; ar << line}}
>    end
>
>    x.report("foreach:") do
>       IO.foreach(file){|line| (lines||=[]) << line.chomp!}
>    end
> end
> #---- end filetest.rb -----
>
> $ seq 10000 > testfile.txt
> $ ruby filetest.rb
> readlines:  0.050000   0.010000   0.060000 (  0.085483)
> inject:  0.130000   0.010000   0.140000 (  0.172144)
> foreach:  0.080000   0.000000   0.080000 (  0.112690)

That's really interesting.  Did you also test with a real world file (a file
with longer lines presumably)?  Interestingly enough, changing the block of
#inject does make it nearly as fast as readlines:

require 'benchmark'

file = "testfile.txt"
Benchmark.bm(7) do |x|
   x.report("readlines:") do
      IO.readlines(file).each {|l| l.chomp!}
   end

   x.report("inject:") do
      File.open(file) {|io| io.inject([]) {
            |ar,line| line.chomp!; ar << line}}
   end

   x.report("inject2:") do
      File.open(file) {|io| io.inject([]) {
            |ar,line| ar << line.chomp!}}
   end


   x.report("foreach:") do
      IO.foreach(file){|line| (lines||=[]) << line.chomp!}
   end

   x.report("foreach2:") do
      lines = []
      IO.foreach(file){|line| lines << line.chomp!}
   end
end



$ seq 10000 > testfile.txt
$ ruby filetest.rb
             user     system      total        real
readlines:  0.078000   0.000000   0.078000 (  0.073000)
inject:  0.094000   0.015000   0.109000 (  0.097000)
inject2:  0.047000   0.016000   0.063000 (  0.072000)
foreach:  0.109000   0.000000   0.109000 (  0.100000)
foreach2:  0.047000   0.000000   0.047000 (  0.048000)

And with more lines:

$ seq 1000000 >| testfile.txt
$ ruby filetest.rb
             user     system      total        real
readlines:  5.625000   0.250000   5.875000 (  5.907000)
inject: 13.985000   0.719000  14.704000 ( 20.221000)
inject2: 13.797000   0.046000  13.843000 ( 14.308000)
foreach: 13.000000   0.032000  13.032000 ( 13.408000)
foreach2:  7.281000   0.031000   7.312000 (  7.455000)


Kind regards

    robert