From: Axel Etzold <AEtzold@...>
Date: 2007-11-04T08:58:12+09:00
Subject: Re: Unicode illegal characters problem

Dear 7stud,

thanks for the effort that you are putting into
solving this problem.
When I thanked about the clarifications Jonathan
gave, I meant that I believe the solution I hoped
to get from the thread I got that code from in
the first place isn't going to work for me as
easily as thought.
I do indeed get a different behaviour for system 
iconv and Ruby iconv, as Jonathan said. 
With respect to the code you sent me, I
get:  

require 'iconv'

s = "caf\xc3\xa9" #(having removed underscores)
p s # => caf\303\251"
ic_translit = Iconv.new('US-ASCII//TRANSLIT', 'UTF-8')
new_s = ic_translit.iconv(s) # => caff`e
p new_s #=> caf?


What system are you on ? 
Mine is Linux OpenSuse 10.2, 64bit, Ruby 1.8.6.
If I had your behaviour on my system, this transliteration
would provide a nice conversion of the accents to Latex,
but now, I think I'll do maybe two dozen gsub lines ...
unless there already is some script that does
a Unicode name to Latex accent conversion, sth. like

small latin letter <lettername> with acute => \'{<lettername>}  ?

Best regards,

Axel 

-------- Original-Nachricht --------
> Datum: Sun, 4 Nov 2007 08:05:49 +0900
> Von: 7stud -- <bbxx789_05ss@yahoo.com>
> An: ruby-talk@ruby-lang.org
> Betreff: Re: Unicode illegal characters problem

> Axel Etzold wrote:
> >> > $KCODE = 'u'
> >> > (from here:
> >> > 
> >> > Thank you very much!
> >> 
> >> 
> >> Your data is not UTF-8, from the error, most likely iso-8859-1 (or 15)
> >> 
> >> man iso_8859-1 shows octal 351 as expected.
> >> 
> >> 351   233   E9     ?     LATIN SMALL LETTER E WITH ACUTE
> >> 
> >> -jh 
> > 
> > Dear Jonathan,
> > 
> > thanks for the hint. You are right. I corrected the encoding
> > of the file I read the text in from,
> > 
> > $KCODE = 'u'
> > require 'iconv'
> > s=IO.readlines("/home/axel/text.txt").to_s
> > p s # => 
> > 'caff?'
> ic_translit = Iconv.new('US-ASCII//TRANSLIT', 'UTF-8')
> > puts ic_translit.iconv(s) # => caff`e
> > 
> > However, now I still get
> > "caff?"  instead of "caff`e" as promised.
> > 
> 
> Try running this code:
> 
> require 'iconv'
> 
> s = "caf\_x_c3\_x_a9" #remove underscores
> p s
> #I see: caf\_303\_251 (without the underscores)
> #\_303\_251 (without the underscores) is the utf-8
> #encoding in octal format.  I really hate that ruby
> #displays octal format instead of hex format!
> 
> 
> ic_translit = Iconv.new('US-ASCII//TRANSLIT', 'UTF-8')
> new_s = ic_translit.iconv(s) # => caff`e
> 
> p new_s  #I see: caf'e
> -- 
> Posted via http://www.ruby-forum.com/.

-- 
Psssst! Schon vom neuen GMX MultiMessenger geh�rt?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger