Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise. #85692

Closed
Carreau mannequin opened this issue Aug 10, 2020 · 19 comments
Closed

codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise. #85692

Carreau mannequin opened this issue Aug 10, 2020 · 19 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker type-bug An unexpected behavior, bug, or error

Comments

@Carreau
Copy link
Mannequin

Carreau mannequin commented Aug 10, 2020

BPO 41520
Nosy @terryjreedy, @vstinner, @ambv, @tacaswell, @Carreau, @csabella, @miss-islington, @tirkarthi
PRs
  • bpo-41520: codeop no longer ignores SyntaxWarning #21838
  • [3.9] bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838) #21840
  • [3.8] bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838) #21841
  • bpo-41520: Fix second codeop regression #21848
  • [3.9] bpo-41520: Fix second codeop regression (GH-21848) #21860
  • [3.8] [3.9] bpo-41520: Fix second codeop regression (GH-21848) #21862
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-08-13.21:16:13.875>
    created_at = <Date 2020-08-10.21:57:55.916>
    labels = ['interpreter-core', 'type-bug', '3.8', '3.9', '3.10', 'release-blocker']
    title = "codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise."
    updated_at = <Date 2020-08-13.21:16:13.874>
    user = 'https://github.com/Carreau'

    bugs.python.org fields:

    activity = <Date 2020-08-13.21:16:13.874>
    actor = 'terry.reedy'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-08-13.21:16:13.875>
    closer = 'terry.reedy'
    components = ['Interpreter Core']
    creation = <Date 2020-08-10.21:57:55.916>
    creator = 'mbussonn'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 41520
    keywords = ['patch']
    message_count = 19.0
    messages = ['375152', '375153', '375154', '375159', '375223', '375230', '375231', '375232', '375236', '375252', '375253', '375273', '375301', '375312', '375322', '375324', '375328', '375329', '375348']
    nosy_count = 8.0
    nosy_names = ['terry.reedy', 'vstinner', 'lukasz.langa', 'tcaswell', 'mbussonn', 'cheryl.sabella', 'miss-islington', 'xtreak']
    pr_nums = ['21838', '21840', '21841', '21848', '21860', '21862']
    priority = 'release blocker'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue41520'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @Carreau
    Copy link
    Mannequin Author

    Carreau mannequin commented Aug 10, 2020

    assuming

        $ cat foo.py
        import warnings
        from codeop import compile_command
        warnings.simplefilter('error', SyntaxWarning)
        res = compile_command('1 is 1\n', symbol='exec')
        print('Res', res)

    On 3.8.0...3.8.4 this correctly raises a SyntaxError:

     python  foo.py
    Traceback (most recent call last):
      File "foo.py", line 5, in <module>
        res = compile_command('1 is 1\n', symbol='exec')
      File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 122, in compile_command
        return _maybe_compile(_compile, source, filename, symbol)
      File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 99, in _maybe_compile
        raise err1
      File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 87, in _maybe_compile
        code1 = compiler(source + "\n", filename, symbol)
      File "/Users/bussonniermatthias/miniconda3/envs/38/lib/python3.8/codeop.py", line 102, in _compile
        return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT)
      File "<input>", line 1
    SyntaxError: "is" with a literal. Did you mean "=="?

    But will silently return None on 3.8.5

    $ python  foo.py
    Res None

    @Carreau Carreau mannequin added 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error labels Aug 10, 2020
    @Carreau
    Copy link
    Mannequin Author

    Carreau mannequin commented Aug 10, 2020

    seem to affect 3.8.4 as well.

    @Carreau
    Copy link
    Mannequin Author

    Carreau mannequin commented Aug 10, 2020

    Potentially due to

    https://bugs.python.org/issue40807
    #20486

    @Carreau Carreau mannequin added 3.9 only security fixes 3.10 only security fixes labels Aug 10, 2020
    @tacaswell
    Copy link
    Mannequin

    tacaswell mannequin commented Aug 11, 2020

    bisecting agrees with Matthias:

    # first bad commit: [c067183] bpo-40807: Show warnings once from codeop._maybe_compile (GH-20486)

    @vstinner vstinner changed the title 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise. codeop: 3.8.5 regression, warnings.simplefilter('error', SyntaxWarning) does not raise. Aug 11, 2020
    @vstinner
    Copy link
    Member

    I proposed a fix: PR 21838. Would you mind to review it?

    @vstinner
    Copy link
    Member

    New changeset 369a1cb by Victor Stinner in branch 'master':
    bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
    369a1cb

    @miss-islington
    Copy link
    Contributor

    New changeset afff51f by Miss Islington (bot) in branch '3.8':
    bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
    afff51f

    @miss-islington
    Copy link
    Contributor

    New changeset 90eff4e by Miss Islington (bot) in branch '3.9':
    bpo-41520: codeop no longer ignores SyntaxWarning (GH-21838)
    90eff4e

    @vstinner
    Copy link
    Member

    Thanks Matthias Bussonnier for the fix. The regression should now be fixed. Also, I converted your reproducer into a regression test.

    @Carreau
    Copy link
    Mannequin Author

    Carreau mannequin commented Aug 12, 2020

    Thanks Matthias Bussonnier for the fix.

    Thank *YOU* for the fix, and the bug report is initially from tcaswell.

    At least on 3.8 branch this is fixed for me.

    Just for completeness, this was discovered as in IPython we try to guess whether "enter" is "insert new line" or "execute", and 1 is 1<enter> would keep adding new lines.

    Much love for the fast turnaround; looking fwd to 3.9

    @vstinner
    Copy link
    Member

    Thanks tcaswell in this case ;-)

    @terryjreedy
    Copy link
    Member

    This 'fix' introduces another regression by undoing the previous fix in bpo-40807 of emitting DeprecationWarning (for '\e', for instance) just once instead of thrice.

    I suspect that the example of Matthias would fail in a debug build (with DeprecationWarnings on) for DeprecationWarning also.

    PR-21848 removes the limitation of the first fix to SyntaxWarning and includes DeprecationWarning in the test for single emission.

    @tirkarthi
    Copy link
    Member

    The commit 369a1cb introduces a change in environment. I haven't checked the other PR though to see if this is fixed.

    ./python.exe -m test --fail-env-changed test_codeop -m test_warning
    0:00:00 load avg: 2.30 Run tests sequentially
    0:00:00 load avg: 2.30 [1/1] test_codeop
    Warning -- warnings.filters was modified by test_codeop
    Before: (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
    After: (4331716096, [('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)], [('error', None, <class 'SyntaxWarning'>, None, 0), ('default', None, <class 'DeprecationWarning'>, '__main__', 0), ('ignore', None, <class 'DeprecationWarning'>, None, 0), ('ignore', None, <class 'PendingDeprecationWarning'>, None, 0), ('ignore', None, <class 'ImportWarning'>, None, 0), ('ignore', None, <class 'ResourceWarning'>, None, 0)])
    test_codeop failed (env changed)

    == Tests result: ENV CHANGED ==

    1 test altered the execution environment:
    test_codeop

    Total duration: 38 ms
    Tests result: ENV CHANGED

    On an unrelated note the test passes with verbose mode though fail-env-changed flag is passed.

    ./python.exe -m test --fail-env-changed test_codeop -m test_warning -vvv
    == CPython 3.10.0a0 (heads/master:c51db0ea40d, Aug 13 2020, 15:47:27) [Clang 10.0.1 (clang-1001.0.46.4)]
    == macOS-10.14.6-x86_64-i386-64bit little-endian
    == cwd: /Users/kasingar/stuff/python/cpython/build/test_python_29945æ
    == CPU count: 8
    == encodings: locale=UTF-8, FS=utf-8
    0:00:00 load avg: 1.53 Run tests sequentially
    0:00:00 load avg: 1.53 [1/1] test_codeop
    test_warning (test.test_codeop.CodeopTests) ... ok

    ----------------------------------------------------------------------

    Ran 1 test in 0.001s

    OK

    == Tests result: SUCCESS ==

    1 test OK.

    Total duration: 58 ms
    Tests result: SUCCESS

    @vstinner
    Copy link
    Member

    The commit 369a1cb introduces a change in environment. (...)

    Right. It's fixed by PR 21848.

    @terryjreedy
    Copy link
    Member

    New changeset c818b15 by Terry Jan Reedy in branch 'master':
    bpo-41520: Fix second codeop regression (GH-21848)
    c818b15

    @vstinner
    Copy link
    Member

    New changeset c818b15 by Terry Jan Reedy in branch 'master':
    bpo-41520: Fix second codeop regression (GH-21848)
    c818b15

    I confirm that this change fixed the "test altered the execution environment" issue.

    @terryjreedy
    Copy link
    Member

    New changeset f24430f by Terry Jan Reedy in branch '3.9':
    [3.9] bpo-41520: Fix second codeop regression (GH-21848)
    f24430f

    @miss-islington
    Copy link
    Contributor

    New changeset a3416c1 by Miss Islington (bot) in branch '3.8':
    [3.9] bpo-41520: Fix second codeop regression (GH-21848)
    a3416c1

    @terryjreedy
    Copy link
    Member

    Lukasz, please cherry-pick the 3.9 commits for this issue
    90eff4e
    f24430f
    into 3.9.0rc2 before release.
    They fix a regression introduced by a bug fix last June.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) release-blocker type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants