From: "Rebhan, Gilbert" <Gilbert.Rebhan@...>
Date: 2007-08-29T21:14:01+09:00
Subject: Detect file encoding utf-8


Hi,

I want to check the file encoding of files in a directory.
Until now i have tried =

# found in an older thread in comp.lang.ruby
class String
  def utf8?
    unpack('U*') rescue return false
  true
  end
end
# found in an older thread in comp.lang.ruby

utf=Array.new
others=Array.new
Dir["Y:/test/**/*.xml"].each do |path|
  open(path) { |f| 
  (f.read.utf8?) ? uts<<path : others<<path
  }
end

and also tried the chardet Library (no ruby documentation included)
like that

require 'UniversalDetector'

utf=Array.new
others=Array.new
Dir["Y:/test/**/*.xml"].each do |path|
 open(path) { |f| 
  UniversalDetector.chardet(f.read) =~ /utf-8/ ?
  uts<<path : others<<path
  }
end
puts utf.join(",")
puts others.join(",")


Are there better / simpler ways ?

Regards, Gilbert