From: "jneen (Jeanine Adkisson) via ruby-core" Date: 2026-02-24T05:41:13+00:00 Subject: [ruby-core:124875] [Ruby Bug#21870] Regexp: Warnings when using slightly overlapping \p{...} classes Issue #21870 has been updated by jneen (Jeanine Adkisson). Having looked through the onigmo code a bit now, I can think of a few ways forward. **a) Simply don't warn on overlapping ctype classes.** I believe this would only involve removing the check on line 1860 from regparse.c. This would preserve a warning for `/[:foo:]/`, as in #1831, as well as maybe rarer situations like `/[a-fb-g]/`. It would *not* warn on cases like `/[a-z\p{Word}]/` or `/[\p{Alnum}\p{Word}]/`. Whether this is a common enough mistake to warrant a warning I'm not entirely sure. I will also check the performance characteristics of these, in case overlapping ranges is a performance issue (which I doubt, but I think it is best to check). **b) Find a way to check if a character class or range completely subsumes another.** I honestly am not sure how I would go about implementing this, as it is a much deeper check which would require a greater understanding of onigmo internals than I have so far. The idea would be to warn on `/[a-z\p{Word}]/` but *not* on e.g. `/[_-z\p{Word}]`, since the range `_-z` contains a character not matched by `\p{Word}`. This would also catch `/[\p{Alnum}\p{Word}]/`. **c) Rethink the overlapping character warning entirely, and (maybe) more specifically target things like `/[:x:]/`.** This would involve warning only if the first and last character of a char class are literal `:`. Similar to (a), it may turn out that repeated characters in classes are not a performance or correctness issue it is worth warning about at all. But this is a judgment I leave to the team. ---------------------------------------- Bug #21870: Regexp: Warnings when using slightly overlapping \p{...} classes https://bugs.ruby-lang.org/issues/21870#change-116534 * Author: jneen (Jeanine Adkisson) * Status: Open * ruby -v: 4.0.1 * Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN ---------------------------------------- ```ruby $VERBOSE = true # warning: character class has duplicated range: /[\p{Word}\p{S}]/ regex = /[\p{Word}\p{S}]/ ``` As far as I can tell this is a perfectly valid and non-redundant set of unicode properties, but I am still being spammed with warnings. Using `/(?:\p{Word}|\p{S})/` is kind of a workaround, but it is slower (see benchmarks below), and also less clear. They do overlap somewhat, but I think the deeper issue is there is not a convenient way to express this without falling back to raw unicode ranges. For a similar example, consider `/[\p{Word}\p{Cf}]/`, which overlap precisely on ZWJ and ZWNJ. Even with this very small overlap, Ruby issues a warning, despite neither class being removable without changing the meaning of the regexp. The regexp is valid and as far as I can tell has no practical issues - Onigmo seems to be capable of intersecting overlapping codepoint ranges. This warning was introduced back in 2009 with #1831, to help surface instances of things like `/[:lower:]/` instead of `/[[:lower:]]/`, but even then the reporter suggested only warning if the class both begins and ends with `:`. Is it appropriate to warn here? Is this a job best left to a static linter like Rubocop, which didn't exist at the time #1831 was opened? Or perhaps would it be better to warn only in the very specific case that #1831 was opened to address? -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- ruby-core@ml.ruby-lang.org To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/