From: Paul van Delst <Paul.vanDelst@...>
Date: 2007-01-25T03:10:06+09:00
Subject: Re: Text file parsing in ruby

Robert Klemme wrote:
> On 24.01.2007 17:02, William James wrote:
>>
>> On Jan 24, 9:33 am, Paul van Delst <Paul.vanDe...@noaa.gov> wrote:

[snip example]

>>>
>>> I find I'm struggling to figure out a tidy way to read these sorts of 
>>> files. If, for
>>> example, I iterate over the lines,
>>>
>>> IO.readlines(file_name).each do |line|
>>>    ...parse the line
>>> end
>>>
>>> How do I take advantage of the fact that the "header" line tells me 
>>> how much actual data
>>> follows before the next header? I.e. I discover that I need to read 5 
>>> point so I read 5
>>> points and the next line that is parsed in the above iteration is the 
>>> next header line.
>>> Sort of short-circuiting the iteration.
>>>
>>> The solution I've come up with so far is to use "sentinel" values 
>>> that flag what is to
>>> come, but it's yuckily kludgy. Any tips from the 'sperts?
>>>
>>
>>
>> open('data1'){|handle|
>>   while header = handle.gets do
>>     header[ /\d+/ ].to_i.times {
>>       p handle.gets
>>     }
>>   end
>> }
> 
> Or test after the fact:
> 
> # untested
> sets = []
> current = nil
> items = nil
> 
> File.foreach('data1') do |line|
>   case line
>     when /Number of data points: (\d+)/
>       raise "Wrong amount" if current && current.size != items
>       items = $1.to_i
>       current = []
>     else
>       current << line.scan(/\d+/).map! {|x| x.to_i}
>   end
> end
> 
> raise "Wrong amount" if current && current.size != items

To all responders, as always, thanks very much. You guys are great. One day I will grok 
this much better (but I have some unlearning to do...)

cheers,

paulv

p.s. Ara, I do use YAML for some things, but I don't always (actually, quite rarely) have 
control of how the file is created. :o(

-- 
Paul van Delst             Ride lots.
CIMSS @ NOAA/NCEP/EMC               Eddy Merckx