From: Stephen Smith <4fires@...>
Date: 2007-03-31T04:31:56+09:00
Subject: Re: text processing

------=_Part_87484_8109122.1175283102366
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Thanks James. Thanks Andy.

So I've cleaned up the regular expression, and I like the simplicity of the
upack message.

But I think that the format Harry and Gary suggested clearly represents the
pattern I'm matching.

Since the pattern may change, and/or we may get other data dumps from this
supplier in the future, I think keeping it represented clearly in one place
will help with maintenance.

Here's what I wound up using:

line = ""
record = ""
oldLog = File.open("filename.txt")
newLog = File.new("filename_formatted.csv", "w")
hdr = "field1, field2, field3,...."
newLog << hdr
arr = oldLog.readlines
arr.each do |line|
record <<
/(.{8})(.{6})(.{15})(.{3})(.{30})(.{4})(.{15})(.{1})(.{12})(.{9})/.match(line).captures.join(',')
<< "\n"
newLogg << record
record = ""
end

+

LOAD DATA INFILE '*/filename.csv*'
INTO TABLE *data_table*
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(field1, field2, field3,...);

And now I can happily munge my monthly data dump to my boss's hearts
content.

Thanks again guys.

I officially love Ruby.

Steve

On 3/30/07, Andrew Stewart <boss@airbladesoftware.com> wrote:
>
>
> On 30 Mar 2007, at 12:59, James Edward Gray II wrote:
> > Here's another option:
> >
> > >> xs, ys, zs = "XXXYYYYYZZ".unpack("A3A5A2")
> > => ["XXX", "YYYYY", "ZZ"]
> > >> xs
> > => "XXX"
> > >> ys
> > => "YYYYY"
> > >> zs
> > => "ZZ"
>
> I like the way your mind works.  Nice!
>
> Regards,
> Andy Stewart
>
>
>

------=_Part_87484_8109122.1175283102366--