[ruby-core:102811] [Ruby master Bug#15718] YAML raises error when dumping strings with UTF32 encoding

Issue #15718 has been updated by jeremyevans0 (Jeremy Evans).

Assignee set to tenderlovemaking (Aaron Patterson)
Status changed from Open to Feedback

I looked into this and ruby YAML uses libyaml, which is a YAML 1.1 implementation.  YAML 1.1 does not support UTF-32 encoding; that isn't supported by the YAML spec until YAML 1.2.  So I think it is reasonable for `YAML.dump` to raise `Encoding::CompatibilityError` for UTF-32 data, and I don't consider this a bug.  Assigning to @tenderlovemaking to make a decision on whether YAML.dump should handle this.

YAML 1.2 is not backwards compatible with YAML 1.1, so I don't think it would be reasonable to switch the YAML library from libyaml to a different library that supports YAML 1.2.  I'm not aware of an existing Ruby library that implements YAML 1.2.

----------------------------------------
Bug #15718: YAML raises error when dumping strings with UTF32 encoding
https://bugs.ruby-lang.org/issues/15718#change-90831

* Author: marcandre (Marc-Andre Lafortune)
* Status: Feedback
* Priority: Normal
* Assignee: tenderlovemaking (Aaron Patterson)
* ruby -v: 2.6.2p247
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
```shell
ruby -r yaml -e "p YAML.dump( ''.force_encoding('UTF-32LE') )"

Traceback (most recent call last):
	4: from -e:1:in `<main>'
	3: from /Users/work/.rvm/rubies/ruby-2.6.1/lib/ruby/2.6.0/psych.rb:513:in `dump'
	2: from /Users/work/.rvm/rubies/ruby-2.6.1/lib/ruby/2.6.0/psych/visitors/yaml_tree.rb:118:in `push'
	1: from /Users/work/.rvm/rubies/ruby-2.6.1/lib/ruby/2.6.0/psych/visitors/yaml_tree.rb:136:in `accept'
/Users/work/.rvm/rubies/ruby-2.6.1/lib/ruby/2.6.0/psych/visitors/yaml_tree.rb:298:in `visit_String': incompatible encoding regexp match (US-ASCII regexp with UTF-32LE string) (Encoding::CompatibilityError)
```

Surprisingly, this works in Ruby 2.4.x, but not in 2.2, 2.3, 2.5 nor 2.6!

---Files--------------------------------
yamldumputf32encodingerror.patch (2.55 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>