From: "ara.t.howard" <ara.t.howard@...>
Date: 2008-11-07T02:25:46+09:00
Subject: Re: how to quickly find a string towards the end of a large io object


On Nov 6, 2008, at 10:18 AM, bwv549 wrote:

> Here's roughly what I'm using right now (sort of an amalgamation of
> everyone's thoughts):
>
> File.open(filename) do |handle|
>      halfway = handle.stat.size / 2
>      handle.seek halfway
>      last_half = handle.read
>      params_start_index = last_half.rindex(my_substring) + halfway
> end
>
> using rindex this takes .17 sec on an 85MB file
> using index it takes  .45 sec on the same file
>
> Those numbers argue that rindex indeed seeks from the end.
>
> Thanks for everyone's help and I will check back to see if there are
> any more thoughts.
>



cfp:~ > cat a.rb
def tail_search io, re, options = {}
   io = open io unless io.respond_to?(:read)
   percent = Float(options['percent']||options[:percent]||0.10)

   buf = ''
   size = io.stat.size
   pagesize = Integer(size * percent)
   pos = 0

   loop do
     pos -= pagesize
     break if pos.abs > size
     io.seek(pos, IO::SEEK_END)
     buf = buf + io.read(pagesize)
     match = re.match(buf)
     return match if match
   end

   return nil
ensure
   io.close rescue nil
end


match = tail_search(__FILE__,  /(key)=(val)/, :percent => 0.02)
p match.to_a if match

p tail_search(__FILE__,  /(key)=(value)/)

__END__
key=val







cfp:~ > ruby a.rb
["key=val", "key", "val"]
nil




you may want to modify to use rindex instead of a regex - up to you.



a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama