From: Phrogz <phrogz@...>
Date: 2007-10-03T02:05:19+09:00
Subject: Re: Binary data, command output, and Ruby

On Oct 1, 9:38 pm, Phrogz <phr...@mac.com> wrote:
> Followup - this does not seem to be a core problem of terminal
> commands returning binary data, or a core failing of Ruby. From my OS
> X box at home:
>
>   Slim2:~/Desktop phrogz$ cat send_bytes.rb
>   print [13,7,129,250,0,70,111,111].map{ |b| b.chr }.join
>
>   Slim2:~/Desktop phrogz$ cat get_bytes.rb
>   result = `ruby send_bytes.rb`
>   p result.length, result
>
>   Slim2:~/Desktop phrogz$ ruby get_bytes.rb
>   8
>   "\r\a\201\372\000Foo"
>
> This is also not a problem with curl (at least on *nix):
>
>   Slim2:~/Desktop phrogz$ curl -s -Ohttp://phrogz.net/tmp/gkhead.jpg
>   Slim2:~/Desktop phrogz$ irb
>   irb(main):001:0> good = IO.read( 'gkhead.jpg' ); good.length
>   => 21443
>   irb(main):002:0> url = 'http://phrogz.net/tmp/gkhead.jpg'
>   => "http://phrogz.net/tmp/gkhead.jpg"
>   irb(main):003:0> test = `curl -s #{url}`; test.length
>   => 21443
>   irb(main):004:0> test == good
>   => true
>
> Tomorrow I'll see which of the above fails back on my Windows box.

Here are the results from Windows. Binary per se doesn't fail, but
using it with curl makes it break eventually.

Any suggestions on how to further pare this down to see if this is a
Ruby-Windows problem, a Windows shell problem, or a Curl-Windows
problem?


c:\>type send_bytes.rb
print [13,7,129,250,0,70,111,111].map{ |b| b.chr }.join

c:\>type get_bytes.rb
result = `ruby send_bytes.rb`
p result.length, result

c:\>ruby get_bytes.rb
8
"\r\a\201\372\000Foo"


c:\>curl -s -O http://phrogz.net/tmp/gkhead.jpg

c:\>irb
irb(main):001:0> good = File.open( 'gkhead.jpg', 'rb' ){ |f| f.read };
good.length
=> 21443

irb(main):002:0> url = 'http://phrogz.net/tmp/gkhead.jpg'
=> "http://phrogz.net/tmp/gkhead.jpg"

irb(main):003:0> test = `curl -s #{url}`; test.length
=> 2010

irb(main):008:0> 0.step( test.length, 100 ){ |i|
irb(main):009:1*   range = i...(i+100)
irb(main):010:1>    if good[ range ] != test[ range ]
irb(main):011:2>       p good[ range ], test[ range ], range
irb(main):012:2>       break
irb(main):013:2>     end
irb(main):014:1> }
"\000\000\000\004\000\000\000\0008BIM\004\032\006Slices
\000\000\000\000m
\000\000\000\006\000\000\000\000\000\000\000\000\000\000\001\276\000\000\001\231\000\000\000\006\000g
\000k\000h\000e\000a\000d
\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\001\231\000\000"
"\000\000\000\004\000\000\000\0008BIM\004$\023\222\vDW$\026\020EG
\377\320\346\177\335q9}K\236:{5C\357L\026\372\330\251\207\261W>
\372\301v\346O\222b\373\027/\276p\310\372\351\370\246\036\314\327~
\366\260\\\t\037\002\236\253\356X\373\267\237\346)\352{\221\221\367I
\352\177\322\2223z`\227\335W"
700...800