From: David Flanagan <david@...>
Date: 2007-10-21T09:39:40+09:00
Subject: patch for String.concat

This is a multi-part message in MIME format.
--------------010604090106040409030708
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I don't think that String.<< currently handles appending codepoints 
correctly.  If I'm running Ruby with -Ku, then I'd expect to be able to 
concat Unicode codepoints to a string that is in all ASCII.  That 
doesn't work currently.

The attached patch is my proposal for fixing this.  If the receiving 
string has ASCII encoding, and the object to be concated is a Fixnum > 
255, then the string has its encoding forced to whatever the primary 
encoding is.  However, if the primary encoding is also ASCII, then it 
raises an Argument Error ("invalid codepoint").

Nobu: wouldn't it be useful if encoding.h declared globals rb_encoding * 
pointers for the ASCII encoding and the primary encoding?  In order to 
do this patch, I had to use this awkward code:

rb_enc_from_index(0)  /* ASCII */
rb_to_encoding(rb_get_primary_encoding() /* primary */

Maybe I just don't know what the best way to get these encoding pointers.

	David

--------------010604090106040409030708
Content-Type: text/plain;
 name="patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="patch"

Index: string.c
===================================================================
--- string.c	(revision 13739)
+++ string.c	(working copy)
@@ -1042,11 +1042,28 @@
 rb_str_concat(VALUE str1, VALUE str2)
 {
     if (FIXNUM_P(str2)) {
+	int c = FIX2INT(str2), pos, len;
 	rb_encoding *enc = rb_enc_get(str1);
-	int c = FIX2INT(str2);
-	int pos = RSTRING_LEN(str1);
-	int len = rb_enc_codelen(c, enc);
 
+	/* 
+	 * If the string is encoded in ASCII and c > 255, then convert
+	 * the string to the primary encoding.  However, if the primary
+	 * encoding is also ASCII, then this is an error
+	 */
+	if (c > 255 && enc == rb_enc_from_index(0)) {
+	    rb_encoding *primary = rb_to_encoding(rb_get_primary_encoding());
+	    if (primary == enc) {
+		rb_raise(rb_eArgError, "invalid codepoint 0x%x", c);
+	    }
+	    else {
+		rb_enc_associate(str1, primary);
+		enc = primary;
+	    }
+	}
+
+	pos = RSTRING_LEN(str1);
+	len = rb_enc_codelen(c, enc);
+
 	if (len == 0) {
 	    rb_raise(rb_eArgError, "invalid codepoint 0x%x", c);
 	}

--------------010604090106040409030708--