From: ara howard <ara.t.howard@...>
Date: 2008-01-09T01:59:02+09:00
Subject: Re: Marshal Pipe


On Jan 7, 2008, at 10:09 PM, Carlos J. Hernandez wrote:

> I'll look again, but I don't think I saw in DRb the simplicity  
> achieved
> by bash as in:
>
> cat source.txt | filter | sort > result.txt
>
> I'm saying cat, filter, and sort could be ruby programs piping Marshal
> data structures.

check out ruby queue (rq) - it uses that paradigm but, instead of  
marshal'd data, it uses yaml which accomplishes the same goal without  
giving up human readability.  for instance one might do (simplified)

rq q query tag==foobar
---
jid: 1
tag: foobar
command: processing_stage_a input

so query is dumping a job object, as yaml.  then you do

!! | rq q update priority=42 -

which is to say use the output of the last command, a ruby object, and  
input that into the next command, which takes a job, or jobs, on stdin  
when '-' is given, and update that job in the queue

you can also do things like

rq q query priority=42 tag=foobar | rq q resubmit -

etc.

the pattern is a good one - but i wouldn't touch marshal data over  
yaml for the commandline with a ten foot pole: one slip and you'll  
blast out chars that will hose the display or disconnect your ssh  
session.  also, yaml provides natural document separators so you can  
embed more than one set in a stream separated by --- which allows for  
chunking of huge output streams

food for thought.

kind regards.



a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama