From: david@...
Date: 2007-12-19T15:34:31+09:00
Subject: Re: Last-minute encoding proposal

Martin Duerst writes: 

> 

> The Ruby model, however, is different. The main purpose of
> both 'transcode' (using that name for the moment for clarity,
> the actual method name is currently 'encode') and 'force_encoding'
> are to change from one encoding to another. In the case of
> 'force_encoding', the 'origin' encoding indeed often is just
> binary, but 'transcode' can be used just as well to convert
> from some (for Ruby) binary stuff such as EBCDIC to e.g. ASCII
> or UTF-8 as to convert from ASCII or UTF-8 to some (for Ruby)
> binary stuff such as EBCDIC or UTF-16. 
> 
> Expressed in other terms, 'transcode' is "change the encoding,
> keep the characters (change the bytes)", whereas 'force_encoding'
> is "change the encoding, keep the bytes (change the characters)".
> Using 'encode' and 'decode' to name this functionality makes it
> very difficult to understand what's actually going on.

My original version of this proposal involved three methods: encode, decode, 
and transcode.  Encode simply stripped an Encoding object from a string, 
converting it to a byte string.  Decode was the current force_encoding: just 
associate an encoding with a byte string.   And transcode was your current 
encode method.  In the purest version of my proposal, that transcode method 
would only work on byte strings and therefore transcoding of bytes was 
completely separate from the issue of interpreting those bytes as characters 
via an Encoding object. This API would probably have been good for 
transcoding a string before writing or after reading from a file.  But it 
was too verbose (involving all three methods) for transcoding from one 
internal encoding to another. 

>
> It is indeed late in the game. My take is that this is an area where
> we need more feedback from practical users. We have had quite a bit of
> discussion on the Japanese list, but it hasn't really been conclusive
> on all points. How encoding that Ruby doesn't support will be handled
> may change in a future version. 
> 

It just doesn't feel like this is finished yet, but I need to figure out how 
to document it in its partially-finished state.