From: dblack@...
Date: 2002-10-12T05:43:51+09:00
Subject: Re: UTF-8 Character Conversion to HTML

Hello --

On Sat, 12 Oct 2002, Ben Schumacher wrote:

> All-
>
> I'm trying to convert some characters from a text file that are
> double-byte characters from Czech and Polish into HTML entities. I have
> a Ruby script that I'm trying to do this with, but when I'm scanning,
> I'm getting several characters, instead of just one. Can somebody help
> me out here? I imagine I'm not doing this in a terribly efficiently
> anyhow, so an optimizations would be appreciated. Here's the text I'm
> converting:
>
> 	Aguarda carregamento da p�gina....
>
> And I'm doing this for the conversion:
>
> str = "Aguarda carregamento da p�gina...."
> str.scan(/./mu) { |c|
> 	v = c.unpack("U.")[0]
> 	if (v > 255)
> 		out += "&\##{v};"
> 	else
> 		out += c
> 	end
> }
> puts str
>
> The output is:
> 	Aguarda carregamento da p᧩na....

Hmmm... I get an error about trying to call + on nil :-) But with a
little tweaking (like initializing "out"), I get what you're
describing.

I'm not an encoding expert, but here's perhaps the beginnings of
a way to do it.  (I've changed 255 to 127, since the � is 225.)

  str = "Aguarda carregamento da p�gina...."
  out = str.unpack("C*").map {|c|
    c > 127 ? "&##{c};" : c.chr
  }.join
  puts out


David

-- 
David Alan Black
home: dblack@candle.superlink.net
work: blackdav@shu.edu
Web:  http://pirate.shu.edu/~blackdav