From: "Ara.T.Howard" <ahoward@...>
Date: 2003-12-18T02:36:58+09:00
Subject: Re: non-english characters

On Wed, 17 Dec 2003, messju mohr wrote:

> yes, it looks like i got confused by the PCRE library which treats \w
> according to the current locale. too-many-languages error. :)

depends on your definition of 'treats' and 'locale' ;-)

-bash-2.05b$ cat /etc/redhat-release
Red Hat Enterprise Linux WS release 3 (Taroon)

-bash-2.05b$ perl -v | head -2 # why so much output!

This is perl, v5.8.0 built for i386-linux-thread-multi

-bash-2.05b$ ruby -v
ruby 1.6.8 (2002-12-24) [i386-linux-gnu]

# BROKEN "TREATMENT" OF LOCALE
-bash-2.05b$ export LANG=en_US.UTF-8
-bash-2.05b$ echo abc | perl -ne 'print if /[^\s]+/'
-bash-2.05b$ echo abc | ruby -ne 'print if /[^\s]+/'
abc

# THIS IS OK 
-bash-2.05b$ export LANG=en_US
-bash-2.05b$ echo abc | perl -ne 'print if /[^\s]+/'
abc
-bash-2.05b$ echo abc | ruby -ne 'print if /[^\s]+/'
abc


definitely need to examine output carefully where regexes and locale are in
effect - probably better off using ruby since matz presumably has more
experience with multibyte chars than 'ol larry!  

-a
-- 

ATTN: please update your address books with address below!

===============================================================================
| EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| STP     :: http://www.ngdc.noaa.gov/stp/
| NGDC    :: http://www.ngdc.noaa.gov/
| NESDIS  :: http://www.nesdis.noaa.gov/
| NOAA    :: http://www.noaa.gov/
| US DOC  :: http://www.commerce.gov/
|
| The difference between art and science is that science is what we
| understand well enough to explain to a computer.  
| Art is everything else.  
|   -- Donald Knuth, "Discover"
|
| /bin/sh -c 'for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done' 
===============================================================================