From: Marcus Andersson <m-lists@...>
Date: 2006-06-14T16:42:18+09:00
Subject: Re: Unicode roadmap?

Yukihiro Matsumoto skrev:
> Hi,
>
> In message "Re: Unicode roadmap?"
>     on Wed, 14 Jun 2006 06:13:03 +0900, Roman Hausner <roman.hausner@gmail.com> writes:
> |In my opinion, Ruby is practically useless for many applications without 
> |proper Unicode support. How a modern language can ignore this issue is 
> |really beyond me.
>
> Define "proper Unicode support" first.
>   
I won't define "proper Unicode support" here.

But there must be a problem somewhere since pure-ruby Ferret doesn't 
support UTF-8. You need to use the c-extension of Ferret to have it 
support UTF-8 (which doesn't work on Windows yet :( ). I don't know if 
that is just a sucky impl of Ferret or if it's Ruby that make it so.

Maybe Dave Balmain can enlighten us why UTF-8 doesn't work in the pure 
Ruby version and what is needed of Ruby to make it work (if it's 
actually Ruby's fault that is)?

My personal belief is that it should just work in a case like this if 
data in is UTF-8 and search strings is UTF-8 without the lib author 
and/or user having to do anything very special to make it work (apart 
from specifying encoding). Am I wrong in this?

Regards,

Marcus