classification
Title: Invalid escape sequence DeprecationWarnings don't trigger by default
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Numerlor, SnoopJeDi, habnabit, ncoghlan, serhiy.storchaka, terry.reedy, vstinner
Priority: normal Keywords:

Created on 2020-04-05 20:29 by Numerlor, last changed 2021-06-03 18:25 by SnoopJeDi.

Files
File name Uploaded Description Edit
gdb_deprecationwarning_session.txt SnoopJeDi, 2021-06-03 18:25
Messages (9)
msg365825 - (view) Author: Numerlor (Numerlor) * Date: 2020-04-05 20:29
The current warning filter seems to filter out the compile time DeprecationWarnings that get triggered on invalid escape sequences:

import warnings

compile("'\d'", "<string>", "eval")
warnings.resetwarnings()
compile("'\d'", "<string>", "eval")


results in one
<string>:1: DeprecationWarning: invalid escape sequence \d
being printed
msg365826 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-04-05 22:11
I get it printed two times.

Actually I get it printed four times: two time when compile the test script (use r"'\d'" or "'\\d'" to get rid of them), and other two times when call compile() inside a script.

$ ./python issue40199.py 
issue40199.py:3: DeprecationWarning: invalid escape sequence \d
  compile("'\d'", "<string>", "eval")
issue40199.py:5: DeprecationWarning: invalid escape sequence \d
  compile("'\d'", "<string>", "eval")
<string>:1: DeprecationWarning: invalid escape sequence \d
<string>:1: DeprecationWarning: invalid escape sequence \d
msg365832 - (view) Author: Numerlor (Numerlor) * Date: 2020-04-06 01:05
In what environment was that ran in?
The issue seems to exist in all the different environments that are easily available to me (win7, win10, linux under docker and on repl.it)
msg366150 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-04-10 20:09
On Win 10, with recently compiled 3.7.7+ and 3.9.0a5+, I get 4 warnings also.

>>> import warnings
>>> compile("'\d'", "<string>", "eval")
<stdin>:1: DeprecationWarning: invalid escape sequence \d
<string>:1: DeprecationWarning: invalid escape sequence \d
<code object <module> at 0x00A65DC0, file "<string>", line 1>
>>> warnings.resetwarnings()
>>> compile("'\d'", "<string>", "eval")
<stdin>:1: DeprecationWarning: invalid escape sequence \d
<string>:1: DeprecationWarning: invalid escape sequence \d
<code object <module> at 0x00A66190, file "<string>", line 1>

Numerior, what 3.8 binaries are you running?  I suspect that this should be closed as 'out of date'.
msg366151 - (view) Author: Numerlor (Numerlor) * Date: 2020-04-10 20:32
The newest 64 bit release from python.org via the executable installer.
msg366191 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-04-11 04:49
OK, verified issue with installed 3.9.0a5, which must have different warning settings.

From a file, 1 warning.
<string>:1: DeprecationWarning: invalid escape sequence \d

Interactively, no warning.

>>> compile("'\d'", "<string>", "eval")
<code object <module> at 0x00000263842DFC90, file "<string>", line 1>
>>>
msg394913 - (view) Author: Aaron Gallagher (habnabit) Date: 2021-06-02 14:25
This is definitely not windows-specific. On macos:

$ python3.9
Python 3.9.4 (default, Apr  5 2021, 01:47:16)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> '\s'
'\\s'
msg394933 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-06-02 18:38
Ah, there is a difference between debug and regular builds. I tested with the debug build.
msg395030 - (view) Author: James Gerity (SnoopJeDi) Date: 2021-06-03 18:25
The cause of DeprecationWarning sometimes [1] not being issued is I believe because in string_parser.c [2] the module is explicitly set to NULL and the filename will be '<string>' or '<stdin>' or somesuch, which eventually that ends up being normalized to something that isn't '__main__'. 

Not sure if this is stating the obvious and I don't see any general solution short of adding a filter for the special filenames, but I caught wind of this in #python on Libera, got curious, and thought I'd share what I learned here. I've also attached a gdb session showing how changing the filename affects this behavior.

Reproducing this in debug/dev contexts is definitely fraught, since the warning behavior is different.

[1] The given compile() sample, at the REPL, when using -c, or when piping input via stdin are the ones I know about
[2] https://github.com/python/cpython/blob/f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970/Parser/string_parser.c#L19-L20
History
Date User Action Args
2021-06-03 18:25:46SnoopJeDisetfiles: + gdb_deprecationwarning_session.txt
nosy: + SnoopJeDi
messages: + msg395030

2021-06-02 18:38:07serhiy.storchakasetversions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.8
nosy: + ncoghlan, vstinner, - paul.moore, tim.golden, zach.ware, steve.dower

messages: + msg394933

components: + Interpreter Core, - Windows
2021-06-02 14:25:06habnabitsetnosy: + habnabit
messages: + msg394913
2020-04-11 07:36:09serhiy.storchakasetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows
2020-04-11 04:49:56terry.reedysetmessages: + msg366191
2020-04-10 20:32:37Numerlorsetmessages: + msg366151
2020-04-10 20:09:27terry.reedysetnosy: + terry.reedy
messages: + msg366150
2020-04-06 01:05:35Numerlorsetmessages: + msg365832
2020-04-05 22:11:42serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg365826
2020-04-05 20:29:24Numerlorcreate