From: "naruse (Yui NARUSE)" Date: 2022-02-10T00:20:33+00:00 Subject: [ruby-core:107539] [Ruby master Feature#18579] Concatenation of ASCII-8BIT strings shouldn't behave differently depending on string contents Issue #18579 has been updated by naruse (Yui NARUSE). Status changed from Open to Rejected The encoding of the resulted string depends "ascii only or not" and "ascii compatibility". The principle of the resulted encoding is * if LHS is ascii only * if RHS is ascii only * LHS's encoding * else if RHS is ascii compatible * RHS's encoding * else (RHS is ascii incompatible) * exception. * else if LHS is ascii compatible * if RHS is ascii only * LHS's encoding * else if RHS is ascii compatible * if LHS's encoding equals to RHS's encoding * LHS's encoding * else * exception * else (RHS is ascii incompatible) * exception. * else if LHS is ascii compatible * if LHS's encoding equals to RHS's encoding * LHS's encoding * else * exception ---------------------------------------- Feature #18579: Concatenation of ASCII-8BIT strings shouldn't behave differently depending on string contents https://bugs.ruby-lang.org/issues/18579#change-96450 * Author: tenderlovemaking (Aaron Patterson) * Status: Rejected * Priority: Normal ---------------------------------------- Currently strings tagged with ASCII-8BIT will behave differently when concatenating depending on the string contents. When concatenating strings the resulting string has the encoding of the LHS. For example: ``` z = a + b ``` `z` will have the encoding of `a` (if the encodings are compatible). However `ASCII-8BIT` behaves differently. If `b` has "ASCII-8BIT" encoding, then the encoding of `z` will sometimes be the encoding of `a`, sometimes it will be the encoding of `b`, and sometimes it will be an exception. Here is an example program: ```ruby def concat a, b str = a + b str end concat "bar", "foo".encode("US-ASCII") # Return value encoding is LHS, UTF-8 concat "bar".encode("US-ASCII"), "foo".b # Return value encoding is LHS, US-ASCII concat "������", "foo".b # Return value encoding is LHS, UTF-8 concat "bar", "bad\376\377str".b # Return value encoding is RHS, ASCII-8BIT. Why? concat "������", "bad\376\377str".b # Exception ``` This behavior is too hard to understand. Usually we think LHS encoding will win, or there will be an exception. Even worse is that string concatenation can "infect" strings. For example: ```ruby def concat a, b str = a + b str end str = concat "bar", "bad\376\377str".b # this worked p str str = concat "������", str # exception p str ``` The first concatenation succeeded, but the second one failed. As a developer it is difficult to find where the "bad string" was introduced. In the above example, the string may have been read from the network, but by the time an exception is raised it is far from where the "bad string" originated. In the above example, the bad data came from like 6, but the exception was raised on line 8. I propose that ASCII-8BIT strings raise an exception if they cannot be converted in to the LHS encoding. So the above program would become like this: ```ruby def concat a, b str = a + b str end concat "bar", "foo".encode("US-ASCII") # Return value encoding is LHS, UTF-8 concat "bar".encode("US-ASCII"), "foo".b # Return value encoding is LHS, US-ASCII concat "������", "foo".b # Return value encoding is LHS, UTF-8 concat "bar", "bad\376\377str".b # Exception <--- NEW!! concat "������", "bad\376\377str".b # Exception ``` I'm open to other solutions, but the underlying issue is that concatenating an ASCII-8BIT string with a non-ASCII-8BIT string is usually a bug and by the time an exception is raised, it is very far from the origin of the string. -- https://bugs.ruby-lang.org/ Unsubscribe: