From: Xavier Noria <fxn@...>
Date: 2009-08-19T07:38:02+09:00
Subject: Re: fromdos dos2unix in ruby

On Tue, Aug 18, 2009 at 11:39 PM, Robert Dober<robert.dober@gmail.com> wrote:

> Yup, I thought my code solved the issue, tell Ruby that a line ends
> with "\n" ( that was tough to type ;) in each and replace a potential
> "\r" before?
> But maybe this does not work on binary files under Windows, no way to
> test, sorry.

The idea is good, but this topic is brittle (though easy when you get
the facts straight).

Problem is on CRLF platforms the I/O system filters out the CR of any
pair CRLF before the string arrives to Ruby land. That is, if you work
in text-mode. In fact that is the definition of text-mode, that the
conversion is on.

When you write in text mode in a CRLF platform, the I/O system
monitors the stream of bytes, and inserts a CR every time he sees an
LF. Unconditionally.

On Unix these conversions do not happen, text-mode and binary-mode are
the same, and Unix uses LF on disk to mean a newline.

And the point is those conversions happen in text-mode *no matter
which is the input record separator*, so in those solution the file
opened for reading should be opened in binary mode anyway. If you
don't do this, a file that has on disk

   \r\r\n

will go up as \r\n on Windows, and that gsubed to \n, so you've lost a
\r that didn't belong to the newline.

In a portable script you have to work in binary mode, and in a
Windows-only script it is enough to read in text-mode and write
verbatim in binary-mode.