From: Brian Candler <b.candler@...>
Date: 2009-12-28T19:36:11+09:00
Subject: Re: ruby 1.9 hates you and me and the encodings we rode in on so just  get used to it.

Benoit Daloze wrote:
> But then I come with things like:
> /Users/benoitdaloze/Library/GlestGame/data/lang/espan>̃<ol.lng
> 
> (The ~ is separated from the n and then is not ñ). The Regexp is acting 
> like
> it is 2 different characters. How to handle that easily? I tried to 
> change
> the script encoding in MacRoman, but it produced an error of bad 
> encoding
> not matching UTF-8.

I don't know what you mean. If Dir.[] tells you that the file name is 
<e> <s> <p> <a> <n> <~> <o> <l> <.> <l> <n> <g>, is that not the true 
filename?

I suggest you try something like this:

  puts "Source encoding: #{"".encoding}"
  puts "External encoding: #{Encoding.default_external}"
  Dir["*.lng"] do |fn|
    puts "Name: #{fn.inspect}"
    puts "Encoding: #{fn.encoding}"
    puts "Chars: #{fn.chars.to_a.inspect}"
    puts "Codepoints: #{fn.codepoints.to_a.inspect}"
    puts "Bytes: #{fn.bytes.to_a.inspect}"
    puts
  end

then post the results for this file here. Then also post what you think 
the true filename is.

Then you can see whether: (1) Dir.[] is returning the correct sequence 
of bytes for the filename or not; and (2) Dir.[] is tagging the string 
with the correct encoding or not.

(This is one of the thousands of cases I did *not* document in 
string19.rb; I did some of the core methods on String, but of course 
every method in every class which either returns a string or accepts a 
string argument needs to document how it handles encodings)

> as output of this script (which is then not able to rename any wrong 
> file,
> because tr! seem to not work either on name) :
> 
> path = ARGV[0] || "/"
> 
> ALLOWED_CHARS = "A-Za-z0-9 %#:$@?!=+~&|'()\\[\\]{}.,\r_-"
> 
> Dir["#{File.expand_path(path)}/**/*"].each { |f|
>     name = File.basename(f)
>     unless name =~ /^[#{ALLOWED_CHARS}]+$/
>         puts File.dirname(f) + '/' + name.gsub(/([^#{ALLOWED_CHARS}]+)/,
> ">\\1<")
> 
>         if name.tr!('éèê', 'e') =~ /^[#{ALLOWED_CHARS}]+$/ # Here it is 
> not
> complete, it is just a test, but it doesn't work even for 'filéname'
>             File.rename(f, File.dirname(f) + '/' + name)
>             puts "\trenamed in #{name}"
>             break
>         end
>     end
> }

What error do you get? Is it failing to match the é at all (tr! returns 
nil), or is an encoding error raised in tr!, or is an error raised by 
File.rename ?
-- 
Posted via http://www.ruby-forum.com/.