From: 7stud -- <bbxx789_05ss@...>
Date: 2007-11-04T07:56:52+09:00
Subject: Re: Unicode illegal characters problem

Axel Etzold wrote:
> -------- Original-Nachricht --------
>> Datum: Sun, 4 Nov 2007 00:55:04 +0900
>> Von: Jonathan Hudson <jh+ruby-lang@daria.co.uk>
>> An: ruby-talk@ruby-lang.org
>> Betreff: Re: Unicode illegal characters problem
> 
>> > $KCODE = 'u'
>> > (from here:
>> > 
>> > Thank you very much!
>> 
>> 
>> Your data is not UTF-8, from the error, most likely iso-8859-1 (or 15)
>> 
>> man iso_8859-1 shows octal 351 as expected.
>> 
>> 351   233   E9     �     LATIN SMALL LETTER E WITH ACUTE
>> 
>> -jh 
> 
> Dear Jonathan,
> 
> thanks for the hint. You are right. I corrected the encoding
> of the file I read the text in from,
> 
> $KCODE = 'u'
> require 'iconv'
> s=IO.readlines("/home/axel/text.txt").to_s
> p s # => 
> 'caff�'
ic_translit = Iconv.new('US-ASCII//TRANSLIT', 'UTF-8')
> puts ic_translit.iconv(s) # => caff`e
> 
> However, now I still get
> "caff?"  instead of "caff`e" as promised.

Another data point:

require 'iconv'

s = "caf\_x_c3\_x_a9"
#The last char is the utf-8 encoding in hex format for 'e' with acute
#I added the underscores so that the encoding won't be rendered
#into the actual character

puts s

#I see cafe where the 'e' is an 'e' with acute, which means my
#display device understands utf-8.

ic_translit = Iconv.new('US-ASCII//TRANSLIT', 'UTF-8')
puts ic_translit.iconv(s)

#I see: caf'e


-- 
Posted via http://www.ruby-forum.com/.