From: Robert Klemme <shortcutter@...>
Date: 2010-11-29T17:12:26+09:00
Subject: Re: Ruby 1.8 vs 1.9

On Sun, Nov 28, 2010 at 6:20 PM, Robert Klemme
<shortcutter@googlemail.com> wrote:
> On 26.11.2010 01:42, David Masover wrote:
>>
>> On Wednesday, November 24, 2010 08:40:22 pm Jörg W Mittag wrote:

>> I mean, yes, you can deliberately build strings out of corrupt data, but
>> if
>> you actually work with complete strings and string concatenation, and you
>> aren't doing crazy JNI stuff, and you aren't digging into the actual bits
>> of
>> the string, I don't see how you can create a truncated string.
>
> Well, you can (see above) but unfortunately it is still valid.  It just
> happens to represent a different sequence.

After reading http://tools.ietf.org/html/rfc2781#section-2.2 I am not
sure any more whether the last statement still holds.  It seems the
presented algorithm can only work reliable if certain code points are
unused.  And indeed checking with
http://www.unicode.org/charts/charindex.html shows that D800 and DC00
are indeed reserved.  Interestingly enough Java's
Character.isDefined() returns true for D800 and DC00:

https://gist.github.com/719100

Cheers

robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/