From: eden li <eden.li@...>
Date: 2007-05-09T11:54:41+09:00
Subject: Re: separate Chinese and English! with Ruby

On May 8, 4:47 pm, Nanyang Zhan <s...@hotmail.com> wrote:
> > If you want to split on those separators, then why not do so
> > explicitly?
>
> > # fill up c as you've done below
> >>> "asdf；asdfasdf".split(/#{c.join('|')}/)
> > => ["asdf", "asdfasdf"]
>
> I don't get it. what this code does?

This code just splits the string at any separator listed in c (no
matter how long it is, byte-wise).  I was guessing at what you were
trying to do, but I understand now.  It looks like you've gotten all
you need now :)

> Chinese character start from 4e00 to 9fa5 at the unicode table, and CJK
> symbols and punctuation range from 3000 to 303f.

There are also a few other ranges, but I'm not sure how popular they
are (from http://www.fileformat.info/info/unicode/block/index.htm):
CJK Compatibility Forms   U+FE30    U+FE4F    (32)
CJK Compatibility Ideographs    U+F900    U+FAFF    (467)
CJK Compatibility   U+3300    U+33FF    (256)
CJK Unified Ideographs Extension A  U+3400  U+4DBF  (6582)
CJK Unified Ideographs Extension B    U+20000   U+2A6DF   (42711)
CJK Compatibility Ideographs Supplement   U+2F800   U+2FA1F   (542)