From: "Wolfgang Nádasi-Donner" <ed.odanow@...>
Date: 2007-10-30T12:13:30+09:00
Subject: Re: Implementation of "guessUTF" method - final questions

Dear Ruby designers, developers, and testers!

I went through the mails and can now come to a conclusion.
It ends up with the following, I will distribute approx.
end of this week (~2007/11/04) via Ruby-Forge a zip file
and a *nix tar file "guess_utf" in the project "guessUTF".
These files will have identical contents, except for line
break characters, which will be "\n" in case of the tar
file and "\r\n" in case of the zip file.

Here are the specification details with some remarks:

1) The file "guess_utf.rb" will contain the method
    "String#check_bom", which tries to find an utf-bom
    at the beginning of the String object.

    I will use the name "guess_utf.rb" for the file,
    because later on more complex methods for guessing the
    Unicode encoding may be added.

2) The return value will be a single value.

    It is not necessary to return the length of recognized
    bom, because it is a constant (utf-8 three bytes,
    utf-16be/le two bytes, and utf-32be/le four bytes). It
    is out of the scope of this program to define additional
    constants for class String. This can be done elsewhere
    if wanted.

3) If the sequence 0xFF 0xFE 0x00 0x00 is found at the
    beginning of the String object it will be flagged as
    "UTF-32LE BOM recognized".

    This sequence is always a valid utf-16le bom followed by
    the codepoint U+0000, so it is not necessary to make a
    special remark.

4) The return values will be Symbol objects or the NilClass
    object "nil".

    It is not possible to return Encoding objects, because
    the Unicode encodings UTF-16, UTF-16BE, UTF-16LE, UTF-32,
    UTF-32BE, and UTF-32LE are not defined for class Encoding
    in Ruby 1.9.1.

5) The return values of "String#check_utf" are:

    nil      - no BOM recognized
    :utf8    - UTF-8 BOM recognized
    :utf16le - UTF-16LE BOM recognized
    :utf16be - UTF-16BE BOM recognized
    :utf32le - UTF-32LE BOM recognized
    :utf32be - UTF-32BE BOM recognized

I hope the naming of the method makes it behaviour clear, and
the return values are usable.

Wolfgang N��dasi-Donner