From: David Flanagan <david@...>
Date: 2007-12-19T05:25:06+09:00
Subject: Last-minute encoding proposal

In my response to Martin's message about encodings supported by 
transcode.c vs. encodings supported by encoding.c, I suggested that if 
the String.encode() method cannot always return a string whose encoding 
method returns a valid Encoding object, then it should always return a 
binary string instead.  I bolstered my argument by saying that this is 
what "encode" means: converting characters into bytes.

It occurs to me now that force_encoding does the opposite: it converts 
bytes into characters, and that therefore, a more appropriate name would 
be "decode".  While this might be a little more confusing at first, it 
is more orthogonal and more logical, and I think it would be a better 
name choice in the long run.

I also think that the fact that there is one set of encodings supported 
by the Encoding class and another set supported for transcoding has the 
potential to be a big source of confusion.  One way to address this is 
to use strings for transcode.c encoding names and Encoding objects for 
encoding.c encodings.

So I propose the following:

1) Rename force_encoding to decode! and define a non-mutating version 
named decode.  The argument must be
an Encoding object or nil.  Encoding names as strings are not allowed.

2) Change the String.encoding method to return nil for byte strings

3) Change the String.encode() method as follows:

   If the encoding of the receiver is non-nil then it expects one
   argument--the destination encoding.

   If the encoding of the receiver is nil, then it must take two
   arguments, the destination encoding and the source encoding.

   If the destination encoding is specified as a string, then the return
   string is a byte string with an encoding of nil.

   If the destination encoding is specified as an Encoding object, then
   the return value is a character string with the specified Encoding
   object associated with it.

   It would look something like this:

   def encode(to, from=nil)

     if !from
       from = self.encoding
       raise ArgumentError if !from
     end

     # Now perform the transcode operation somehow
     result = transcode(self, to, from)

     if (to.is_a? Encoding) result.decode!(to)

     result
   end

The benefits of this proposal are:

a) It gives us nice, logical method names encode and decode
b) It clarifies the distinction between byte strings and character 
strings: byte strings have a nil encoding
c) It helps (a little) to distinguish between Encoding objects that can 
be used to interpret bytes as characters and encoding names that can be 
used as transcoding targets but which are not otherwise supported.

I assume that encoding issues have been discussed heavily in Japanese on 
ruby-dev already, and I'm well aware that it is late in the process, but 
I'm throwing these ideas out there anyway. My apologies if these ideas 
have already been considered and rejected.

	David