From: unbewusst.sein@... (=?ISO-8859-1?Q?Une_B=E9vue?=)
Date: 2010-03-11T22:50:06+09:00
Subject: Re: reading an UTF-8 encoded file

Brian Candler <b.candler@pobox.com> wrote:

> Use 'puts' instead of 'p' and it may work. That is, I suspect 
> String#inspect is doing some mangling.
> 
> You really should look at your postings in ruby-forum:
> http://www.ruby-forum.com/topic/205792
> 
> Wherever you say ruby 1.9 is giving the 'wrong' output it is correct,
> and where you say ruby 1.8 is giving the 'right' output it is wrong. I
> have a suspicion that there is a mismatch between the file content and
> the terminal.
> 
> What if you just type "cat /Users/yt/dev/Signature/signatures.txt" at
> the terminal?

i got the correct chars :
zsh-% cat /Users/yt/dev/Signature/signatures.txt
-- 
� Un banquier est toujours en libert� provisoire � 
(Henri Poincar� )

...
-- 
� Pour ceux qui vont chercher midi � quatorze heures, 
la minute de v�rit� risque de se faire longtemps attendre. � 
(Pierre Dac)

> 
> > accentuated chars are correct now, notice i have to use "puts" instead
> > of "p" to get the chars otherwise i got the unicode code as
> > "v\303\251rit\303\251".
> 
> Yes, String#inspect in ruby 1.8 will mangle all values over 128 into 
> escaped form. String#inspect in ruby 1.9 behaves differently, and 
> doesn't always mangle them.
> 
> However, I just noticed 'macruby' in your scripts. Are you actually 
> running MacRuby, or genuine Matz Ruby Interpreter 1.9 ? If it's macruby
> all bets are off - I thought it was a completely different interpreter
> written from scratch. I have no Mac here to compare behaviour with, and
> I have no idea what variation of 1.9 encoding rules MacRuby has 
> implemented.
> 
> In particular, I'm surprised that your program sees strings tagged as
> "US-ASCII" rather than "UTF-8" when you explicitly opened the file with
> external encoding of UTF-8. This makes me very suspicious of your actual
> ruby platform.

right now, that's to say using puts in place of p, i get the right
chars.
But those strings are still taged by "US-ASCII"...

> Try adding this line to your code to get info about the Ruby platform:
> p Object.constants.grep(/RUBY/).map { |n| [n, Object.const_get(n)] }

it seems to be an "old" Ruby 1.9.0 :

[[:RUBY_VERSION, "1.9.0"], [:RUBY_RELEASE_DATE, "2008-06-03"],
[:RUBY_PLATFORM, "universal-darwin10.0"], [:RUBY_PATCHLEVEL, 0],
[:RUBY_REVISION, 0], [:RUBY_DESCRIPTION, "MacRuby version 0.5 (ruby
1.9.0) [universal-darwin10.0, x86_64]"], [:RUBY_COPYRIGHT, "MacRuby -
Copyright (C) 2007-2008 Apple Inc."], [:RUBY_ENGINE, "macruby"],
[:RUBY_ARCH, "x86_64"], [:MACRUBY_VERSION, "0.5"], [:MACRUBY_REVISION,
"svn revision 3380 from
http://svn.macosforge.org/repository/ruby/MacRuby/branches/0.5"]]

thanks again !
-- 
� L'essence m�me du g�nie, c'est de mettre en pratique 
les id�es les plus simples. � 
(Charles Peguy)