From: Ashley Moran <work@...>
Date: 2006-08-30T04:16:35+09:00
Subject: Evading the limit of a pipe's standard input

Hi

I'm trying to write a tool that generates a really long SQL script  
and passes it to psql, but I've hit a problem because my script is  
too long to be sent to the stdin of psql.  This is the first time  
I've run into the limit so it had me scratching my head for a while.   
I've tried a load of tricks, even putting the lines in an array, eg:

   MAX_SUCCESSFUL_TIMES = 3047

   query = ["BEGIN WORK;"]
   (MAX_SUCCESSFUL_TIMES + 1).times do
     query << "INSERT INTO people (name) VALUES ('Fred');"
   end
   query << "COMMIT;"

   IO.popen("psql -U test test","r+") do |pipe|
     query.each { |statement| pipe.puts statement }
   end

but it still fails when the total length of commands exceeds the  
limit (which by experiment I've found to be 128K on Mac OS X, hence  
the specific number of times above).

What's the best solution to this.  I would like to stick to inter- 
process communication, and avoid temporary files and rewriting it to  
use DBD, if possible.  Or are they my only options?

Thanks
Ashley