From: Kjell Olsen <kjell@...11.net>
Date: 2007-02-27T10:50:11+09:00
Subject: open-uri and utf8

I'm trying to make requests to google translate [http://google.com/ 
translate_t] to translate words through a little ruby script. I can't  
get open-uri to work on URL's with accented characters (����� etc).  
Just calling open() gives:

URI::InvalidURIError: bad URI(is not URI?): http://google.com/ 
translate_t?langpair=en|fr&text=�lire
         from /usr/local/lib/ruby/1.8/uri/common.rb:432:in `split'
         from /usr/local/lib/ruby/1.8/uri/common.rb:481:in `parse'
         from /usr/local/lib/ruby/1.8/open-uri.rb:29:in `open'
         from (irb):2
         from :0

URI.encode()ing the url breaks the characters down into gunk (� => % 
C3%A9) which the translator doesn't understand.

I've tried fooling with $KCODE and switching to net/http, neither to  
any avail. Anyone have a hand to give me? I'll paste the code in full  
below in case anyone wants to see it.

-kjell

--- translate.rb ------------

#!/usr/local/bin/ruby
%w[rubygems open-uri hpricot active_support readline].each {|lib|  
require lib}
$KCODE = 'u'

# can't handle �/�/utf8 chars because open-uri won't let us have  
special chars in a query, nor will net/http
class GoogleTranslator
   attr_reader :doc
   @@langs = 'fr|en'

   def initialize(text, langs=@@langs)
     @text, @langs = text.chomp, langs
     @doc = Hpricot(open(URI.encode("http://google.com/translate_t? 
langpair=#{@langs}&text=#{@text}")))
   end

   def result
     @result ||= @doc.search('#result_box').inner_html
     write_history_line
     @result
   end

   def write_history_line # keep a file with all the translations,  
just for kicks.
     `echo "#{"[#{@langs}]\t#{@text}\t\t->\t#{@result}"}" | cat >> '/ 
Users/kjell/mess/2007/03/translation-history.txt'`
   end

   class << self
     include Readline

     def prompt; "[#{@@langs}] > "; end
     def interact!
       while text = readline(prompt, true)
         (text =~ /lang: (.*)/) ? @@langs = $1 : puts("# 
{GoogleTranslator.new(text).result}") # either switch languages or  
spit out a translation
       end
     end
   end
end

ARGV[0] ? puts(GoogleTranslator.new(*ARGV[0..1]).result) :  
GoogleTranslator.interact! # if called with an argument, translate  
that argument; else set up for interaction