From: Charles Oliver Nutter <charles.nutter@...>
Date: 2007-12-31T03:02:20+09:00
Subject: Re: multibyte strings & bucket-of-bytes efficiency under 1.9.0

khaines@enigo.com wrote:
> On Sat, 29 Dec 2007, Brent Roman wrote:
> 
>>
>> The first application I benchmarked on 1.9 was a logger that did
>> a lot of "bucket of bytes" type string manipulations.  The whole
>> app ended up running 5% slower under 1.9 than it did under 1.8.6.
>> I'd been suspecting the encoded strings were largely responsible
>> for this.  Your experience seems to confirm this.
>>
>> I've been thinking about implementing a:
>>
>> class ByteString < String
> 
> If it is based on the multibyte String, though, I expect that it'd still 
> exhibit the terrible performance with certain methods.
> 
> IMHO, it'd make more sense in the other direction:
> 
> class String < ByteString
> 
> Looking at the code for string.c, though, it'd probably be a whole lot 
> easier to just make them distinct and independent.

I talked with Matz about this last Spring, and we agreed that there were 
certain aspects of encodings that might be represented well with 
multiple types. However he wanted to keep a single type outwardly 
presented to Ruby for everything.

I think perhaps a big problem here is that Strings can change encoding. 
This means that encoding has to constantly be checked, even in the case 
of a ByteString, since it could be changed later. If encoding was 
immutable, and the only way to get a new encoding on a string were to 
construct a new string (which could COW the underlying byte array, 
naturally) then each string object could be *just* the logic for one 
encoding.

We're debating a few different ways to implement M17N strings in JRuby, 
but avoiding any performance degradation for common string operations 
will be a pretty strict requirement.

- Charlie