Re: xmlrpc and charset=utf-8

On 6/19/06, Sean Russell <[e-mail]> wrote:
> I first sent this from the wrong email account, so if that post somehow makes
> its way onto the list, then please forgive the repitition.
>
> On Monday 19 June 2006 13:35, Kazuhiro NISHIYAMA wrote:
> > > Was this ever addressed?  I vote for both a default of
> > > utf8 and an accessor method.
> >
> > http://www.zvon.org/tmRFC/RFC3023/Output/chapter8.html#sub5
> >
> > | This example shows text/xml with the charset parameter omitted.
> > | In this case, MIME and XML processors MUST assume the charset is
> > | "us-ascii"
>
> This is interesting.  It seems to be at odds with the XML specification, which
> says:
>
>         http://www.w3.org/TR/2006/PER-xml-20060614/#charencoding
>
>         > In the absence of information provided by an external transport protocol
>         > (e.g. HTTP or MIME), it is a fatal error for an entity including an
>         > encoding declaration to be presented to the XML processor in an encoding
>         > other than that named in the declaration, or for an entity which begins
>         > with neither a Byte Order Mark nor an encoding declaration to use an
>         > encoding other than UTF-8. Note that since ASCII is a subset of UTF-8,
>         > ordinary ASCII entities do not strictly need an encoding declaration.
>
> I read this to say that XML documents, in the absence of both external
> encoding information or an XML declaration, must be assumed to be UTF-8.
> RFC3023 appears to be saying that XML documents default to US-ASCII.

You are correct in your interpretation of the XML spec, and I agree
that mentioned XMLRPC C library appears to be the flawed
implementation. The XML spec reads:

Although an XML processor is required to read only entities in the
UTF-8 and UTF-16 encodings, it is recognized that other encodings are
used around the world, and it may be desired for XML processors to
read entities that use them. In the absence of external character
encoding information (such as MIME headers), parsed entities which are
stored in an encoding other than UTF-8 or UTF-16 MUST begin with a
text declaration (see 4.3.1 The Text Declaration) containing an
encoding declaration....

And RFC 3023 states that charset parameter of the text/xml
registration is strongly recommended. The following description of the
charset parameter is straight from RFC 3023:

Although listed as an optional parameter, the use of the charset
parameter is STRONGLY RECOMMENDED, since this information can be
used by XML processors to determine authoritatively the character
encoding of the XML MIME entity.  The charset parameter can also
be used to provide protocol-specific operations, such as charset-
based content negotiation in HTTP.  "utf-8" [RFC2279] is the
recommended value, representing the UTF-8 charset.  UTF-8 is
 supported by all conforming processors of [XML].

Cheers,
Dom