Re: Problems with \M-x in utf-8 encoded strings

And each_char shouldn't throw an index error...no?

Am 18.11.2007 um 14:01 schrieb Wolfgang N=C3=A1dasi-Donner =
<ed.odanow@wonado.d=20
e>:

> Hi!
>
> I recognized a problem when using special character modifying =20
> sequences
> (\M-x and \C-x) in utf-8 encoded strings...
>
> t =3D "a\M-aa"
> puts t.encoding                     # =3D> <Encoding:UTF-8>
> puts t.length                       # =3D> 2
> puts t.bytesize                     # =3D> 3
> t.each_byte{|b|print("%X " %b)}     # =3D> 61 E1 61
> puts
> t.each_char{|c|print("%X" % c.ord)} # =3D> utf8manipul.rb:7:in =20
> `each_char':
>                                    # =3D> index out of range =20
> (IndexError)
>                                    # =3D>         from =
utf8manipul.rb:7:
>                                    # =3D> in `<main>'
>                                    # =3D> 61
>
> ..., because "\M-a" will produce 0xE1, which is a possible starting =20=

> byte
> for a three byte utf-8 encoded character.
>
> Another utf-8 destroying situation occurs with "\C-=E2=82=AC"...
>
> t =3D "=E2=82=AC\C-=E2=82=AC"
> puts t.encoding                      # =3D> <Encoding:UTF-8>
> puts t.length                        # =3D> 4
> puts t.bytesize                      # =3D> 6
> t.each_byte{|b|print("%X " %b)}      # =3D> E2 82 AC 82 82 AC
> puts
> t.each_char{|c|print("%X " % c.ord)} # =3D> 20AC 82 82 AC
>
> ..., because it produces the leading byte 0x82 for an utf-8 encoding
> sequence, which is invalid. The resulting String is an ill-formed
> utf-8 sequence.
>
> My proposal is either not to allow these special modifiers for other
> encodings than Ascii (which might be complicated), or to enforce
> Ascii encoding for Strings, that contain these special modifiers.
>
> Wolfgang N=C3=A1dasi-Donner
>
>