From: duerst@... Date: 2019-04-16T10:05:41+00:00 Subject: [ruby-core:92304] [Ruby trunk Bug#15764] Whitespace and control characters should not be permitted in tokens Issue #15764 has been updated by duerst (Martin D�rst). There may be a question as to what to do with encodings other than UTF-8. I see three possibilities: 1) Ignore (i.e. leave as is), because who still uses them? 2) Use a rule based on character properties such as letter and digit that are supported in these encodings 3) Convert to Unicode and check there I would be okay with any one of these. ---------------------------------------- Bug #15764: Whitespace and control characters should not be permitted in tokens https://bugs.ruby-lang.org/issues/15764#change-77647 * Author: BatmanAoD (Kyle Strand) * Status: Open * Priority: Normal * Assignee: matz (Yukihiro Matsumoto) * Target version: * ruby -v: * Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN ---------------------------------------- As of Ruby 2.5.1p57, it appears that all valid Unicode code-points above 128 are permitted in tokens. This includes whitespace and control characters. This was demonstrated here: https://gist.github.com/qrohlf/7045823 I have attached the raw download from the above gist. The issue has been discussed on StackOverflow: https://stackoverflow.com/q/34455427/1858225 I would say this is arguably a bug, but I am marking this ticket as a "feature" since the current behavior could be considered by-design. ---Files-------------------------------- helloworld.rb (543 Bytes) -- https://bugs.ruby-lang.org/ Unsubscribe: