From: Andrew Timberlake <andrew@...>
Date: 2009-01-02T17:29:38+09:00
Subject: Re: simple garbage problem

------=_Part_216907_24225630.1230885022957
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On Fri, Jan 2, 2009 at 10:10 AM, <cchayden.nyt@gmail.com> wrote:

> When I run the program:
>
> STDIN.each_line { |line| line.split("\t") }
>
> with a big input file, the process size grows rapidly, reaching 1G in
> less than 1 minute.
> If I let it go, it continues to grow at this rate until the whole
> system fails.
> This is ruby 1.8.6 on fedora 9.
>
> Is this a known problem?
> Is there some way to work around it?
>
>
I haven't done much input processing in ruby but I've used this with no
problems on large files:

while line = STDIN.gets
  ...
end

Just check that your file line breaks and the line breaks being used by
gets/each_line are the same.
Also make sure you actually have line breaks (sometimes you just gotta
check)

Andrew
http://ramblingsonrails.com

------=_Part_216907_24225630.1230885022957--