From: Robert Klemme <shortcutter@...>
Date: 2007-09-20T15:55:05+09:00
Subject: Re: How to parse a unicode url?

On 20.09.2007 07:19, 7stud -- wrote:
> Dan The man wrote:
>> I would really like to be able to do the following. Is this even
>> possible?
>>
>> Thanks,
>> nerdytenor
>>
>> uri = URI.parse('http://www.h�ren.com') # not a real url (that I know
>> of)
>> URI::InvalidURIError: bad URI(is not URI?): http://www.h�ren.com
>>         from /usr/lib/ruby/1.8/uri/common.rb:432:in `split'
>>         from /usr/lib/ruby/1.8/uri/common.rb:481:in `parse'
>>         from (irb):26

There is no such thing as a Unicode URL.  The RFC for URI and URL 
specify the charset as 7Bit ASCII AFAIK.

The legal form of that URL is this: http://www.xn--hren-5qa.com/

See IDNA for details, for example:
http://de.wikipedia.org/wiki/IDNA

Quick searching revealed this - maybe it can help:
http://rubyforge.org/pipermail/idn-discuss/2005-September/000000.html

> You can do this:
> 
> require "uri"
> 
> url = "http://www.h�ren.co"�
> enc_url = URI.encode(url)
> puts enc_url
> 
> 
> to get this:
> 
> http://www.h%C3%B6ren.co
> 
> which according to wikipedia here:
> 
> http://en.wikipedia.org/wiki/Percent-encoding
> 
> is a legal uri.  But when I do this:
> 
> 
> require "uri"
> 
> url = "http://www.h�ren.co"
> enc_url = URI.encode(url)
> puts enc_url
> 
> uri = URI.parse(enc_url)
> 
> I get this:
> 
> http://www.h%C3%B6ren.co
> /usr/lib/ruby/1.8/uri/generic.rb:194:in `initialize': the scheme http 
> does not accept registry part: www.h%C3%B6ren.co (or bad hostname?) 
> (URI::InvalidURIError)
>         from /usr/lib/ruby/1.8/uri/http.rb:46:in `initialize'
>         from /usr/lib/ruby/1.8/uri/common.rb:484:in `new'
>         from /usr/lib/ruby/1.8/uri/common.rb:484:in `parse'
>         from r3test.rb:7
> 
> 
> which as far as I can tell means that URI.parse() is broken.

I don't think so.  There are invalid characters in the domain name (as 
the exception indicates).

Kind regards

	robert