classification
Title: Remove deprecated re features
Type: enhancement Stage: resolved
Components: Library (Lib), Regular Expressions Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: barry, ezio.melotti, mrabarnett, ned.deily, python-dev, r.david.murray, serhiy.storchaka
Priority: Keywords: patch

Created on 2016-05-15 18:38 by serhiy.storchaka, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
re_remove_deprecated.patch serhiy.storchaka, 2016-05-15 18:38 review
re-sub-allow-unknown-escapes.patch serhiy.storchaka, 2016-12-06 10:25 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (16)
msg265641 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-05-15 18:38
Proposed patch removes following deprecated re features:

* Three unused not documented functions:  isident(), isdigit() and isname(). They were deprecated since Python 3.3 (issue14462).

* '\' + ASCII character now is error if this combination is not defined. This allows to add new control combinations without breaking compatibility. This was deprecated since Python 3.5 (issue23622).

* Support for re.LOCALE with string patterns or with re.ASCII. This was deprecated since Python 3.5 (issue22407).
msg268222 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-11 16:15
New changeset 09d1af3fe332 by Serhiy Storchaka in branch 'default':
Issue #27030: Unknown escapes consisting of ``'\'`` and ASCII letter in
https://hg.python.org/cpython/rev/09d1af3fe332
msg268223 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-06-11 16:16
Thanks Jim for the review.
msg268224 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-11 16:24
New changeset 8ed3880e94e5 by Serhiy Storchaka in branch 'default':
Issue #27030: The re.LOCALE flag now can be used only with bytes patterns.
https://hg.python.org/cpython/rev/8ed3880e94e5
msg268334 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-06-12 06:28
New changeset a2482e805dff by Martin Panter in branch '3.5':
Fix buggy RE “\parrot_example.py”, uncovered by Issue #27030
https://hg.python.org/cpython/rev/a2482e805dff

New changeset be193f8dbe4c by Martin Panter in branch 'default':
Issue #27030: Merge RE fix from 3.5
https://hg.python.org/cpython/rev/be193f8dbe4c

New changeset c5411cfa0bd3 by Martin Panter in branch '2.7':
Fix buggy RE “\parrot_example.py”, uncovered by Issue #27030
https://hg.python.org/cpython/rev/c5411cfa0bd3
msg281485 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-11-22 14:41
FWIW, this breaks Mailman 3.1 (and probably 2.1)
msg281487 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-11-22 14:42
Specifically, point #2; undefined combinations of \ + ASCII becoming an error.
msg281491 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-22 16:13
Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings in 3.5. And now they emit warnings even just in string literals (issue27364). If Mailman use undefined escape combinations, it could suffer from issue27364 too.
msg281496 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-11-22 18:19
On Nov 22, 2016, at 04:13 PM, Serhiy Storchaka wrote:

>Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings
>in 3.5. And now they emit warnings even just in string literals
>(issue27364). If Mailman use undefined escape combinations, it could suffer
>from issue27364 too.

No, I think this is a valid bug/regression.

The Mailman code is basically trying to do this:

    p = re.compile('%\d*d')
    p.sub(r'\s*\d+\s*', some_string)

And so we get the error:

sre_constants.error: bad escape \s at position 0

But this directly contradicts the documentation for re.sub():

"... if it is a string, any backslash escapes in it are processed. That is, \n
is converted to a single newline character, \r is converted to a carriage
return, and so forth. Unknown escapes such as \& are left alone."

Clearly \s is not being left alone, so this is a real regression.
msg281498 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-22 18:55
This part of the documentation was just overlooked. Issue28450 is opened for this.
msg281503 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-22 19:34
If you insist I could revert converting warnings to errors (only in replacement string or all?) in 3.6. But I think they should left errors in 3.7. The earlier we make undefined escapes the errors, the earlier we can define new special escape sequences without confusing users. It is bad if the escape sequence is valid in two Python versions but has different meaning.
msg281511 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-11-22 19:55
There is still the argument that we shouldn't break 2.7 compatibility unnecessarily until 2.7 is out of maintenance.  That is: warnings are good, removals are bad.  (I haven't read through this issue, so I may be off base.)
msg282514 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-06 10:25
Here is a patch that partially reverts changes of issue27030. It allows using unknown escapes in re.sub() replacement template, but they are deprecated and will be errors in 3.7.

If this is good to you Barry, and Ned allows, it can be committed in 3.6 only.

But this could delay adding support of new escapes (like \xXX, \uXXXX, etc) in replacement templates.
msg282556 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-12-06 16:56
It is unfortunate that the deprecation note did not explicitly mention replacement templates as well as regexp patterns.  While there are good arguments to be made for either case, I think it makes more sense to treat the replacement template as following the rules for strings rather than for regexps which would argue for making the proposed change to cause only a deprecation warning in templates for 3.6 - although I'm not happy about making such a change at the last minute.  Serhiy, please push to 3.6 for rc1.  There should also be a note in the What's New for 3.6 document.
msg282557 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-12-06 17:25
New changeset 1b162d6e3d01 by Serhiy Storchaka in branch '3.6':
Issue #27030: Unknown escapes in re.sub() replacement template are allowed
https://hg.python.org/cpython/rev/1b162d6e3d01

New changeset 5904d2ced3d8 by Serhiy Storchaka in branch 'default':
Merge documentation for issue #27030 from 3.6.
https://hg.python.org/cpython/rev/5904d2ced3d8
msg282572 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2016-12-06 22:26
Thanks, Serhiy, for reverting the error handling for 3.6.0 (in 3.6.0rc1).  I'm going to mark this as closed.  Issue28450 is still open at the moment regarding the documentation and possible 3.7 changes.
History
Date User Action Args
2017-03-31 16:36:29dstufftsetpull_requests: + pull_request1021
2016-12-06 22:26:24ned.deilysetstatus: open -> closed
priority: release blocker ->
versions: + Python 3.7
messages: + msg282572

resolution: fixed
stage: commit review -> resolved
2016-12-06 17:25:46python-devsetmessages: + msg282557
2016-12-06 16:56:29ned.deilysetassignee: ned.deily -> serhiy.storchaka
messages: + msg282556
stage: patch review -> commit review
2016-12-06 10:25:34serhiy.storchakasetstatus: closed -> open
files: + re-sub-allow-unknown-escapes.patch

assignee: serhiy.storchaka -> ned.deily
resolution: fixed -> (no value)
stage: resolved -> patch review
messages: + msg282514
priority: normal -> release blocker
2016-11-22 19:55:51r.david.murraysetnosy: + r.david.murray
messages: + msg281511
2016-11-22 19:34:29serhiy.storchakasetmessages: + msg281503
2016-11-22 19:23:28barrysetpriority: release blocker -> normal
status: open -> closed
2016-11-22 18:55:47serhiy.storchakasetmessages: + msg281498
2016-11-22 18:19:59barrysetpriority: normal -> release blocker
status: closed -> open

nosy: + ned.deily
2016-11-22 18:19:23barrysetmessages: + msg281496
2016-11-22 16:13:07serhiy.storchakasetmessages: + msg281491
2016-11-22 14:48:19pitrousetnosy: - pitrou
2016-11-22 14:42:47barrysetmessages: + msg281487
2016-11-22 14:41:47barrysetnosy: + barry
messages: + msg281485
2016-06-12 06:28:46python-devsetmessages: + msg268334
2016-06-11 16:24:49python-devsetmessages: + msg268224
2016-06-11 16:16:31serhiy.storchakasetstatus: open -> closed
messages: + msg268223

assignee: serhiy.storchaka
resolution: fixed
stage: patch review -> resolved
2016-06-11 16:15:31python-devsetnosy: + python-dev
messages: + msg268222
2016-05-15 18:38:16serhiy.storchakacreate