From: Cameron McBride <cameron.mcbride@...>
Date: 2007-02-04T08:22:36+09:00
Subject: Re: Manipulating CSV files over SSH

On 2/3/07, Logan Capaldo <logancapaldo@gmail.com> wrote:
> On Sun, Feb 04, 2007 at 07:47:31AM +0900, Drew Olson wrote:
> > All -
> >
> > I have a nice little script that takes a large CSV document and splits
> > it into 65,000 line chunks which are readable in Excel. However, I
> > currently have a .csv file which I'd like to split that is sitting on a
> > unix box. I have ssh access to the box, however the file is quite big
> > and I'd like to avoid downloading the whole file. Is there a simple way
> > to modify my script such that I use the ssh library to access the
> > document and perform the splitting over the network?
> >
> > Thanks in advance,
> > Drew
> >
> ssh user@host -x 'cat large.csv' | your_script
> should work pretty well for this.

That is essentially downloading the entire file, which he doesn't want to do.

Wow, hot topic.  I was going to suggest what Jan Svitok did:  The
easiest thing to do is to upload and run the ruby script on the unix
box.

If the script takes a while to run, you might want to check out
"screen" (but that is a bit off topic - ping me personally if you need
more info on this than you can google).

Cameron