From: unbewusst.sein@... (=?ISO-8859-1?Q?Une_B=E9vue?=)
Date: 2010-03-11T15:40:07+09:00
Subject: Re: reading an UTF-8 encoded file

Brian Candler <b.candler@pobox.com> wrote:

> Are you in irb, or running code in a .rb file? Are you using "puts" or
> are you looking at the string values as returned by irb, after the =>
> prompt?
> 
> In either case, show your actual code. Beware that things behave 
> strangely in irb with 1.9. 

first, thanks for your reply )))

i'm not using irb rather an rb file kaunched from Terminal

here is the code (ruby 1.9) :

------------------------------------------------------------------------
#! /usr/local/bin/macruby
# encoding: utf-8

SIGNATURES_FILE = "/Users/yt/dev/Signature/signatures.txt"

open(SIGNATURES_FILE, "r:UTF-8") do |file|
  p    file.internal_encoding
  file.each do |line|
    p [line.encoding.name, line]
  end
end
open(SIGNATURES_FILE) do |file|
  p    file.internal_encoding
  file.each do |line|
    p [line.encoding.name, line]
  end
end
------------------------------------------------------------------------

resulting in :
zsh-% ./essai_macruby.rb
nil
["US-ASCII", "-- \n"]
["US-ASCII", "�� Un banquier est toujours en libert�� provisoire �� \n"]
["US-ASCII", "(Henri Poincar�� )\n"]

...

["US-ASCII", "la minute de v��rit�� risque de se faire longtemps
attendre. �� \n"]
["US-ASCII", "(Pierre Dac)\n"]
nil
["US-ASCII", "-- \n"]
["US-ASCII", "�� Un banquier est toujours en libert�� provisoire �� \n"]
["US-ASCII", "(Henri Poincar�� )\n"]
[
...

["US-ASCII", "la minute de v��rit�� risque de se faire longtemps
attendre. �� \n"]
["US-ASCII", "(Pierre Dac)\n"]
zsh-% 


then, both methods (with and without "r:UTF-8") see the file as being of
US-ASCII although they are really UTF-8 encoded.

now the "equivalent" test using ruby 1.8.* :

------------------------------------------------------------------------
#! /usr/bin/env ruby

SIGNATURES_FILE = "/Users/yt/dev/Signature/signatures.txt"

open(SIGNATURES_FILE) do |file|
  file.each do |line|
    puts line
  end
end
------------------------------------------------------------------------

run from Term :

zsh-% ./essai.rb
-- 
� Un banquier est toujours en libert� provisoire � 
(Henri Poincar� )

...

� Pour ceux qui vont chercher midi � quatorze heures, 
la minute de v�rit� risque de se faire longtemps attendre. � 
(Pierre Dac)


accentuated chars are correct now, notice i have to use "puts" instead
of "p" to get the chars otherwise i got the unicode code as
"v\303\251rit\303\251".
-- 
� L'essence m�me du g�nie, c'est de mettre en pratique 
les id�es les plus simples. � 
(Charles Peguy)