From: "jeremyevans0 (Jeremy Evans) via ruby-core" <ruby-core@...>
Date: 2023-03-24T17:25:52+00:00
Subject: [ruby-core:112999] [Ruby master Bug#19417] Regexp \p{Word} and [[:word:]] do not match Unicode Other_Number character

Issue #19417 has been updated by jeremyevans0 (Jeremy Evans).





naruse (Yui NARUSE) wrote in #note-3:

> The document is wrong. The definition of `word` is defined in Unicode� Technical Standard #18 UNICODE REGULAR EXPRESSIONS.

> https://unicode.org/reports/tr18/#word



I've updated my pull request to match the description in the standard linked by @naruse.  @janosch-x or @naruse, could you review?



----------------------------------------

Bug #19417: Regexp \p{Word} and [[:word:]] do not match Unicode Other_Number character

https://bugs.ruby-lang.org/issues/19417#change-102528



* Author: ObjectBoxPC (Philip Chung)

* Status: Open

* Priority: Normal

* ruby -v: 3.2.0

* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

----------------------------------------

According to the [documentation for Regexp](https://ruby-doc.org/3.2.0/Regexp.html), `\p{Word}` and `[[:word:]]` both match a character in one of the following Unicode general categories: Letter, Mark, Number, Connector_Punctuation. However, neither matches U+00B2, which is in the Other_Number category (which is a subcategory of Number).



``` ruby

puts "Ruby version: %s" % RUBY_VERSION

puts "\p{Word} matches? %s" % /\p{Word}/u.match?("\u00B2")

puts "[[:word:]] matches? %s" % /[[:word:]]/u.match?("\u00B2")

puts "Is a Number charater? %s" % /\p{Number}/u.match?("\u00B2")

puts "Is an Other_Number character? %s" % /\p{Other_Number}/u.match?("\u00B2")

```



Expected output:



```

Ruby version: 3.2.0

p{Word} matches? true

[[:word:]] matches? true

Is a Number charater? true

Is an Other_Number character? true

```



Actual output:



```

Ruby version: 3.2.0

p{Word} matches? false

[[:word:]] matches? false

Is a Number charater? true

Is an Other_Number character? true

```



I notice that the [upstream Onigmo library doc](https://github.com/k-takata/Onigmo/blob/master/doc/RE) defines the `[[:word:]]` class as "Letter | Mark | Decimal_Number | Connector_Punctuation", meaning that it only matches certain number characters (which would exclude U+00B2). I am not sure how `\p{Word}` is defined though. But perhaps the documentation needs to be changed?







-- 

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/