This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: 3.10.0b2 traces with-exit before the break that caused the exit
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Mark.Shannon, nedbat, pablogsal
Priority: release blocker Keywords: 3.10regression, patch

Created on 2021-06-03 10:05 by nedbat, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 26513 merged Mark.Shannon, 2021-06-03 13:50
PR 26516 merged Mark.Shannon, 2021-06-03 15:56
Messages (8)
msg394990 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-03 10:05
Python 3.10 now traces back to with statements when exiting the with block. When the exit is a break statement, the with exit is visited before the break statement is.  This seems confusing.

--- 8< -----------------------------------------

import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

def doit():
    for i in range(2):
        with open("test", "w") as f:
            a = 13
            b = 14
            break
    c = 16

print(sys.version)
sys.settrace(trace)
doit()

--- 8< -----------------------------------------

3.10.0b2 (default, Jun  3 2021, 05:27:13) [Clang 12.0.0 (clang-1200.0.32.29)]
call 10: def doit():
line 11:     for i in range(2):
line 12:         with open("test", "w") as f:
line 13:             a = 13
line 14:             b = 14
line 12:         with open("test", "w") as f:
line 15:             break
line 16:     c = 16
retu 16:     c = 16



Shouldn't we get a trace for line 15 (break), then line 12 (with-exit), then line 15 again, then line 16?
msg394991 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-03 10:16
(I'm not sure whether to create other issues for further details)

I'm also seeing a return in a with will trace withexit/return for a plain "return" statement, but return/withexit/return for a return with a value ("return 17").

I would expect that a statement causing an exit from a with block would always be traced before the with-exit.
msg394994 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-03 12:13
For context, this behavior was introduced in https://bugs.python.org/issue43933
msg394995 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-03 12:18
Why this occurs:

with cm: 
    A
    break

translates to something like:

ex = cm.__exit__; cm.__enter__()  # with cm
    A
    ex(...)
    goto loop_end   # break

So, the break is traced after the exit call.

However, this doesn't seem consistent with try-finally statements which trace any break/continue/return before the finally block.
msg395007 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-03 15:46
New changeset 937cebc93b4922583218e0cbf0a9a14705a595b2 by Mark Shannon in branch 'main':
bpo-44298: Fix line numbers for early exits in with statements. (GH-26513)
https://github.com/python/cpython/commit/937cebc93b4922583218e0cbf0a9a14705a595b2
msg395033 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-03 18:57
New changeset cea0585b7939b487d7089f9d473f495264e8a491 by Mark Shannon in branch '3.10':
[3.10] bpo-44298: Backport #26513 to 3.10 (#26516)
https://github.com/python/cpython/commit/cea0585b7939b487d7089f9d473f495264e8a491
msg395038 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-06-03 19:59
Ned, can you confirm that this works for you? I am closing the issue, but if something is missing, please, reopen it and we will look into it :)
msg395052 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-03 21:41
Thanks for the quick turnaround, this works!
History
Date User Action Args
2022-04-11 14:59:46adminsetgithub: 88464
2021-06-03 21:41:17nedbatsetmessages: + msg395052
2021-06-03 19:59:00pablogsalsetstatus: open -> closed
resolution: fixed
messages: + msg395038

stage: patch review -> resolved
2021-06-03 18:57:38pablogsalsetmessages: + msg395033
2021-06-03 15:56:39Mark.Shannonsetpull_requests: + pull_request25111
2021-06-03 15:46:07Mark.Shannonsetmessages: + msg395007
2021-06-03 13:50:03Mark.Shannonsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request25109
2021-06-03 12:18:54Mark.Shannonsetpriority: normal -> release blocker

nosy: + pablogsal
messages: + msg394995

keywords: + 3.10regression
stage: needs patch
2021-06-03 12:13:21Mark.Shannonsetmessages: + msg394994
2021-06-03 10:16:36nedbatsetmessages: + msg394991
2021-06-03 10:05:02nedbatcreate