From: Michael Linfield Date: 2007-08-16T13:08:26+09:00 Subject: Re: grep a csv? M. Edward (Ed) Borasky wrote: > Michael Linfield wrote: >> >> ### this sadly only returned an output of => [] >> >> any ideas? >> >> Thanks! > > OK ... first of all, define "huge" and what are your restrictions? Let > me assume the worst case just to get started -- more than 256 columns > and more than 65536 rows and you're on Windows. :) > > Seriously, though, if this is a *recurring* use case rather than a > one-shot "somebody gave me this *$&%^# file and wants an answer by 5 PM > tonight!" use case, I'd load it into a database (assuming your database > doesn't have a column count limitation larger than the column count in > your file, that is) and then hook up to it with DBI. But if it's a > one-shot deal and you've got a command line handy (Linux, MacOS, BSD or > Cygwin) just do "grep blah1 huge-file.csv > temp-file.csv". Bonus points > for being able to write that in Ruby and get it debugged before someone > who's been doing command-line for years types that one-liner in. :) lol, alright lets say the senario will be in the range of 20k - 70k lines of data. no more than 20 columns and i wanna avoid using command line to do this, because yes in fact this will be used to process more than one datafile which i hope to setup in optparse to have a command line arg that directs the prog to the file. also i wanted to for the meantime not have to throw it on any database...avoiding DBI for the meanwhile. But an idea flew through my head a few minutes ago....what if i did this -- res = [] res << File.readlines('filename.csv').grep(/Blah1/) #thanks chris ran into a small demeaning problem. this shoves all that grep'd data into 1 element lol... res[1] => nil ...its all shoved into res[0] id hope to fix this with a simple do statement but a little confusion hit me while doing that with a readline command. and by shoving this into an array will i still be able to single out columns of data. if not then how would i shove the grep data into a second csv file, doing this all inside ruby of course, no command line program > output.csv :) -- Posted via http://www.ruby-forum.com/.