Title: Nail down and document the behavior of range expressions in RE character classes
Type: behavior Stage:
Components: Documentation, Regular Expressions Versions:
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, ezio.melotti, mrabarnett, zwol
Priority: normal Keywords:

Created on 2018-07-19 19:03 by zwol, last changed 2018-07-19 19:05 by zwol.

Messages (2)
msg321963 - (view) Author: Zack Weinberg (zwol) * Date: 2018-07-19 19:03
The documentation of the semantics of range expressions in regular expression character classes is not precise enough.  All it says is

    Ranges of characters can be indicated by giving two characters and separating them by a '-', for example [a-z] will match any lowercase ASCII letter [... more examples, none involving non-ASCII characters]

In testing it seems that the behavior is simply to expand the range to a set of characters by numeric code point, e.g. '[ᄀ-ፚ]' will match any single character whose ord() is in between ord('ᄀ') and ord('ፚ') (inclusive).  If that is the intended behavior, I would like the documentation to explicitly say so.  If that is _not_ the intended behavior, I would like to know what the intended behavior actually is, and for both the code and the documentation to be changed to reflect the intent.

(I think expansion by numeric code point makes sense and is probably what most existing programs want, but this is a contended issue in the context of POSIX regular expressions, e.g. some C libraries try (not always successfully) to make [0-9] match all of the characters that Python's \d matches, so it's not "obvious".)
msg321964 - (view) Author: Zack Weinberg (zwol) * Date: 2018-07-19 19:05
Also, whether or not the current behavior is the intended behavior, I think programmers would appreciate an explicit statement of whether or not it might change in the future.
Date User Action Args
2018-07-19 19:05:07zwolsetmessages: + msg321964
2018-07-19 19:03:20zwolcreate