From: Don Wood <dwood@...>
Date: 2009-04-17T02:40:18+09:00
Subject: Re: Tokenizing a large file

Robert Klemme wrote:
> 2009/4/16 Ryan Davis <ryand-ruby@zenspider.com>:
>> % irb
>>>> h = {}
>> => {}
>>>> h["key"] = 42
>> => 42
>>>> h.keys.map { |k| k.frozen? }
>> => [true]
>>
>> hashes dupe and freeze string keys to prevent them from being mutated while
>> hash keys.
> 
> Only if they are not frozen yet.
> 
> irb(main):001:0> h = {}
> => {}
> irb(main):002:0> s = "abc"
> => "abc"
> irb(main):003:0> h[s] = s
> => "abc"
> irb(main):004:0> s = "bar".freeze
> => "bar"
> irb(main):005:0> h[s] = s
> => "bar"
> irb(main):006:0> h
> => {"abc"=>"abc", "bar"=>"bar"}
> irb(main):007:0> h.each {|kv| p kv.map {|x| x.object_id}}
> [134954550, 134972840]
> [134951170, 134951170]
> => {"abc"=>"abc", "bar"=>"bar"}
> 
> Do you now know why I did it the way I did?
> 
> Cheers
> 
> robert

Thanks Robert,

I see what you did there.  This looks like the perfect solution for 
finding duplicate strings quickly.  I don't want to assume that tokens 
don't span lines but, combining this with Caleb's suggestion of using 
the sequence gem, I should have all I need to drastically cut my memory 
footprint.
-- 
Posted via http://www.ruby-forum.com/.