From: "Wolfgang Nádasi-Donner" <ed.odanow@...>
Date: 2007-10-29T12:11:27+09:00
Subject: Implementation of "guessUTF" method - final questions

Dear Ruby designers, developers, and testers!

As recently announced in topic "Hopefully Complete List of Possible
Encoding Specifications - Existing Ones", I provide the method "guessBOM"
for discussion now. If it may be helpful for user that the method may be
in Ruby 1.9.1 standard distribution, than I would like to discuss details
before finishing the method (it isn't as complicated and large).

The original intention for a module "UTFchecker" containing methods "utf8?",
"utf16?", "utf32?" , and "utfBOM?" was to identify incomplete and corrupted
textual data, which I usually expect to receive for analysis. The method
"utfBOM?" is different from the other insofar, as it should only give a hint
what Unicode encoding might be used in the file, if it is Unicode encoded
at all. The others are very slow and much more complicated, because they
support several possibilities in reporting and other reactions.

For a possible usage inside standard distribution I suggest to ship the
BOM checking method without the others and make it as handy and small as
possible to avoid overhead. The other methods will be a different project
in future.

My proposal:

1) First of all the method will receive the name "guessUTF", and I build
    it as an instance method in an extension of class "String". The
    method will be in the file "guessutf.rb". Is this acceptable for you?

    For me it seem the most natural way to ask a String object, if it's
    first bytes give some hints for a possible Unicode encoding.

2) At present the method returns a number between 0 and 5 with the
    following meaning:

    0 - no BOM recognized (either there is no BOM or the String object
        is too short)
    1 - UTF-8 BOM recognized
    2 - UTF-16LE BOM recognized
    3 - UTF-16BE BOM recognized
    4 - UTF-32LE BOM recognized or UTF-16LE with 0x0000 after BOM
    5 - UTF-32BE BOM recognized

    I don't know if you prefer different return codes, maybe Symbol
    objects or predefined constants. It will be a simple change if
    there are other preferences.

3) The method is commented and readable structered about 150 Lines
    of simple Ruby code (it's a finite state machine). In addition
    there is a small test "guessutftest.rb", which isn't complicated
    at all. It belongs to the project. No additional data is provided, but
    documentation can be produced via "rdoc". Is this the recommended
    procedure?

There are four questions left:

1) I'm developing the sources on Windows, so the files contain usually "\r\n"
    instead of "\n" for newlines. Should I change it to "\n" before delivering
    the programs?

2) I usually use ZIP files in the moment for putting files together (typical
    for Windows environments). Should I use a different package format for
    delivery? - Which one?

3) I've written in the first comments in the two files the line

    # License: Ruby license

    Is this the recommended text for such program files?

4) Is it the recommended procedure to distribute the file(s) via
    Ruby-Forge? - If not, what else is recommended?

This are my final questions. It would be nice to have answers in the next
days, because I can finish the whole action in approx. one week.

Thank you in advance for comments.

Wolfgang N��dasi-Donner