This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise.
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: cheryl.sabella, lukasz.langa, mbussonn, miss-islington, pablogsal, tcaswell, terry.reedy, vstinner, xtreak
Priority: release blocker Keywords: patch

Created on 2020-08-10 21:57 by mbussonn, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 21838 closed vstinner, 2020-08-12 11:26
PR 21840 merged miss-islington, 2020-08-12 12:53
PR 21841 merged miss-islington, 2020-08-12 12:54
PR 21848 merged terry.reedy, 2020-08-12 17:50
PR 21860 merged terry.reedy, 2020-08-13 17:41
PR 21862 merged miss-islington, 2020-08-13 18:21
Messages (19)
msg375152 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2020-08-10 21:57
assuming 

    $ cat foo.py
    import warnings
    from codeop import compile_command

    warnings.simplefilter('error', SyntaxWarning)
    res = compile_command('1 is 1\n', symbol='exec')
    print('Res', res)

On 3.8.0...3.8.4 this correctly raises a SyntaxError:

 python  foo.py
Traceback (most recent call last):
  File "foo.py", line 5, in <module>
    res = compile_command('1 is 1\n', symbol='exec')
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 122, in compile_command
    return _maybe_compile(_compile, source, filename, symbol)
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 99, in _maybe_compile
    raise err1
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 87, in _maybe_compile
    code1 = compiler(source + "\n", filename, symbol)
  File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 102, in _compile
    return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
  File "<input>", line 1
SyntaxError: "is" with a literal. Did you mean "=="?



But will silently return None on 3.8.5

$ python  foo.py
Res None
msg375153 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2020-08-10 21:58
seem to affect 3.8.4 as well.
msg375154 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2020-08-10 22:05
Potentially due to 

https://bugs.python.org/issue40807
https://github.com/python/cpython/pull/20486
msg375159 - (view) Author: Thomas Caswell (tcaswell) * Date: 2020-08-11 00:30
bisecting agrees with Matthias:

# first bad commit: [c067183605cf84bb1a246635f52827251d0476f8] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)
msg375223 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-12 11:28
I proposed a fix: PR 21838. Would you mind to review it?
msg375230 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-12 12:53
New changeset 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 by Victor Stinner in branch 'master':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/369a1cbdee14d9f27356fb3a8bb21e4fde289d25
msg375231 - (view) Author: miss-islington (miss-islington) Date: 2020-08-12 13:12
New changeset afff51fc09993dff1693aacb440221314b163409 by Miss Islington (bot) in branch '3.8':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/afff51fc09993dff1693aacb440221314b163409
msg375232 - (view) Author: miss-islington (miss-islington) Date: 2020-08-12 13:13
New changeset 90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad by Miss Islington (bot) in branch '3.9':
bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
https://github.com/python/cpython/commit/90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad
msg375236 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-12 13:21
Thanks Matthias Bussonnier for the fix. The regression should now be fixed. Also, I converted your reproducer into a regression test.
msg375252 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2020-08-12 15:14
> Thanks Matthias Bussonnier for the fix. 

Thank *YOU* for the fix, and the bug report is initially from tcaswell. 

At least on 3.8 branch this is fixed for me.

Just for completeness, this was discovered as in IPython we try to guess whether "enter"  is "insert new line" or "execute", and `1 is 1<enter>` would keep adding new lines.

Much love for the fast turnaround; looking fwd to 3.9
msg375253 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-12 15:28
Thanks tcaswell in this case ;-)
msg375273 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-08-12 17:55
This 'fix' introduces another regression by undoing the previous fix in issue 40807 of emitting DeprecationWarning (for '\e', for instance) just once instead of thrice.

I suspect that the example of Matthias would fail in a debug build (with DeprecationWarnings on) for DeprecationWarning also.

PR-21848 removes the limitation of the first fix to SyntaxWarning and includes DeprecationWarning in the test for single emission.
msg375301 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2020-08-13 11:02
The commit 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 introduces a change in environment. I haven't checked the other PR though to see if this is fixed.

./python.exe -m test --fail-env-changed test_codeop -m test_warning
0:00:00 load avg: 2.30 Run tests sequentially
0:00:00 load avg: 2.30 [1/1] test_codeop
Warning -- warnings.filters was modified by test_codeop
  Before: (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
  After:  (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('error', None, <class 'SyntaxWarning'>, None, 0), ('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
test_codeop failed (env changed)

== Tests result: ENV CHANGED ==

1 test altered the execution environment:
    test_codeop

Total duration: 38 ms
Tests result: ENV CHANGED


On an unrelated note the test passes with verbose mode though fail-env-changed flag is passed.

./python.exe -m test --fail-env-changed test_codeop -m test_warning -vvv
== CPython 3.10.0a0 (heads/master:c51db0ea40d, Aug 13 2020, 15:47:27) [Clang 10.0.1 (clang-1001.0.46.4)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/kasingar/stuff/python/cpython/build/test_python_29945æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.53 Run tests sequentially
0:00:00 load avg: 1.53 [1/1] test_codeop
test_warning (test.test_codeop.CodeopTests) ... ok

----------------------------------------------------------------------

Ran 1 test in 0.001s

OK

== Tests result: SUCCESS ==

1 test OK.

Total duration: 58 ms
Tests result: SUCCESS
msg375312 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-13 14:24
> The commit 369a1cbdee14d9f27356fb3a8bb21e4fde289d25 introduces a change in environment. (...)

Right. It's fixed by PR 21848.
msg375322 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-08-13 17:18
New changeset c818b15fa59039de67022c29085d439fa5d3ef95 by Terry Jan Reedy in branch 'master':
bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/c818b15fa59039de67022c29085d439fa5d3ef95
msg375324 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-08-13 17:22
> New changeset c818b15fa59039de67022c29085d439fa5d3ef95 by Terry Jan Reedy in branch 'master':
> bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/c818b15fa59039de67022c29085d439fa5d3ef95

I confirm that this change fixed the "test altered the execution environment" issue.
msg375328 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-08-13 18:21
New changeset f24430f1542ea2768793b48704ae2d4e241892ae by Terry Jan Reedy in branch '3.9':
[3.9] bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/f24430f1542ea2768793b48704ae2d4e241892ae
msg375329 - (view) Author: miss-islington (miss-islington) Date: 2020-08-13 18:39
New changeset a3416c13b51af25675e175d4ffe07d8b925e7dbf by Miss Islington (bot) in branch '3.8':
[3.9] bpo-41520: Fix second codeop regression (GH-21848)
https://github.com/python/cpython/commit/a3416c13b51af25675e175d4ffe07d8b925e7dbf
msg375348 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-08-13 21:16
Lukasz, please cherry-pick the 3.9 commits for this issue
90eff4ed4445a0fa9d8cdf74c0f86c50ed510dad
f24430f1542ea2768793b48704ae2d4e241892ae
into 3.9.0rc2 before release.
They fix a regression introduced by a bug fix last June.
History
Date User Action Args
2022-04-11 14:59:34adminsetnosy: + pablogsal
github: 85692
2020-08-13 21:16:13terry.reedysetstatus: open -> closed
resolution: fixed
messages: + msg375348

stage: patch review -> resolved
2020-08-13 18:39:03miss-islingtonsetmessages: + msg375329
2020-08-13 18:21:56miss-islingtonsetpull_requests: + pull_request20989
2020-08-13 18:21:40terry.reedysetmessages: + msg375328
2020-08-13 17:41:08terry.reedysetstage: needs patch -> patch review
pull_requests: + pull_request20987
2020-08-13 17:22:42vstinnersetmessages: + msg375324
2020-08-13 17:18:57terry.reedysetmessages: + msg375322
2020-08-13 14:24:48vstinnersetmessages: + msg375312
2020-08-13 11:02:22xtreaksetnosy: + xtreak
messages: + msg375301
2020-08-12 17:55:03terry.reedysetstatus: closed -> open
priority: release blocker


nosy: + cheryl.sabella
messages: + msg375273
resolution: fixed -> (no value)
stage: resolved -> needs patch
2020-08-12 17:50:13terry.reedysetnosy: + terry.reedy

pull_requests: + pull_request20975
2020-08-12 15:28:10vstinnersetpriority: release blocker -> (no value)

messages: + msg375253
2020-08-12 15:14:27mbussonnsetmessages: + msg375252
2020-08-12 13:21:27vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg375236

stage: patch review -> resolved
2020-08-12 13:13:12miss-islingtonsetmessages: + msg375232
2020-08-12 13:12:09miss-islingtonsetmessages: + msg375231
2020-08-12 12:54:01miss-islingtonsetpull_requests: + pull_request20969
2020-08-12 12:53:58vstinnersetmessages: + msg375230
2020-08-12 12:53:54miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20968
2020-08-12 11:28:11vstinnersetmessages: + msg375223
2020-08-12 11:26:03vstinnersetkeywords: + patch
nosy: + vstinner

pull_requests: + pull_request20966
stage: patch review
2020-08-11 15:56:51vstinnersettitle: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise. -> codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise.
2020-08-11 15:55:36vstinnersetpriority: normal -> release blocker
2020-08-11 05:30:31xtreaksetnosy: + lukasz.langa
2020-08-11 00:30:03tcaswellsetnosy: + tcaswell
messages: + msg375159
2020-08-10 22:33:57mbussonnsetversions: + Python 3.9, Python 3.10
2020-08-10 22:05:33mbussonnsetmessages: + msg375154
2020-08-10 21:58:47mbussonnsetmessages: + msg375153
2020-08-10 21:57:55mbussonncreate