From: TAKAHASHI Masayoshi <maki@...>
Date: 2001-11-11T16:20:32+09:00
Subject: [ruby-talk:24861] Re: XML libraries (Re:  Re: ruby and webservices)

Hi,

Tobias Reif <tobiasreif@pinkjuice.com> wrote:
> I tried to generate SVG with NQXML; the generated output is no XML, thus can not be parsed by any SVG viewer. (workarounds are nof un)
> I tried to include data from an RSS feed; as soon as an NCR like &#233; occurs in the RSS, NQ generates no more output. (can that be true? I hope
> it's my mistake)

Hmmm...
I made a patch for nqxml-1.1.1(see below). This patch has a problem:
it's correct only for the document's encoding is UTF-8.

IMHO, default encoding of XML parser in Ruby should be UTF-8.
Because XML is in Unicode world, not ISO-8859-* nor EUC world
(unfortunately for me). And Ruby's regex doesn't support
UTF-16.
So, if the parser support only one encoding, it should be UTF-8,
and documents in other encoding should be converted to UTF-8.

Is it good solution?

TAKAHASHI 'Maki' Masayoshi     E-mail: maki@open-news.com

--- utils.rb~   Wed Jul 11 12:05:40 2001
+++ utils.rb    Sun Nov 11 15:38:52 2001
@@ -23,8 +23,8 @@
     # replaced by the characters they represent.
     def NQXML.replaceCharacterRefs(str)
        return nil if str.nil?
-       copy = str.gsub(/&\#(\d+);/n) { $1.to_i.chr }
-       return copy.gsub(/&\#x([0-9a-f]+);/ni) { $1.hex.chr }
+       copy = str.gsub(/&\#(\d+);/n) { [$1.to_i].pack("U*") }
+       return copy.gsub(/&\#x([0-9a-f]+);/ni) { [$1.hex].pack("U*") }
     end
 
     # Returns a new string with all of the special character refs (for

--- tokenizer.rb~       Thu Aug  2 22:13:08 2001
+++ tokenizer.rb        Sun Nov 11 15:45:57 2001
@@ -289,10 +289,10 @@
                result << $`
                val = val[pos .. -1]
                if val =~ /\A&#x([0-9a-f]*);/ni
-                   result << $1.hex.chr
+                   result << [$1.hex].pack("U*")
                    val = $'
                elsif val =~ /\A&#([0-9]*);/n
-                   result << $1.to_i.chr
+                   result << [$1.to_i].pack("U*")
                    val = $'
                elsif val =~ /\A&([^&;]*);/n
                    ref = $1