classification
Title: match/case statements trace incorrectly in 3.10.0b4
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brandtbucher Nosy List: Mark.Shannon, ammar2, brandtbucher, chaburkland, jack__d, miss-islington, nedbat, pablogsal
Priority: release blocker Keywords: patch

Created on 2021-07-11 21:02 by nedbat, last changed 2021-07-26 00:04 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27346 merged chaburkland, 2021-07-25 04:42
PR 27356 merged miss-islington, 2021-07-25 23:42
Messages (11)
msg397264 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-07-11 21:02
Some simple match/case statements show incorrect tracing.  Below is the code to run, as well as the output.  Output lines with initial stars are incorrect: they incorrectly indicate that case bodies are executing when they are not. Sorry for the bulk here, I wanted to give you all the cases I had.

-- 8< -------------------------------------
import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

def match_with_default():
    for command in ["huh", "go home", "go n"]:
        print(command)
        match command.split():
            case ["go", direction] if direction in "nesw":
                match = f"go: {direction}"
            case ["go", _]:
                match = "no go"
            case _:
                match = "default"
        print(match)

def match_with_wildcard():
    for command in ["huh", "go home", "go n"]:
        print(command)
        match command.split():
            case ["go", direction] if direction in "nesw":
                match = f"go: {direction}"
            case ["go", _]:
                match = "no go"
            case x:
                match = f"default: {x}"
        print(match)

def match_without_wildcard():
    match = None
    for command in ["huh", "go home", "go n"]:
        print(command)
        match command.split():
            case ["go", direction] if direction in "nesw":
                match = f"go: {direction}"
            case ["go", _]:
                match = "no go"
        print(match)

print(sys.version)
sys.settrace(trace)
match_with_default()
match_with_wildcard()
match_without_wildcard()
-- 8< -------------------------------------
Output:

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 10: def match_with_default():
  line 11:     for command in ["huh", "go home", "go n"]:
  line 12:         print(command)
  huh
  line 13:         match command.split():
  line 14:             case ["go", direction] if direction in "nesw":
  line 15:                 match = f"go: {direction}"
  line 16:             case ["go", _]:
* line 17:                 match = "no go"
  line 19:                 match = "default"
  line 20:         print(match)
  default
  line 11:     for command in ["huh", "go home", "go n"]:
  line 12:         print(command)
  go home
  line 13:         match command.split():
  line 14:             case ["go", direction] if direction in "nesw":
  line 16:             case ["go", _]:
  line 17:                 match = "no go"
  line 20:         print(match)
  no go
  line 11:     for command in ["huh", "go home", "go n"]:
  line 12:         print(command)
  go n
  line 13:         match command.split():
  line 14:             case ["go", direction] if direction in "nesw":
  line 15:                 match = f"go: {direction}"
  line 20:         print(match)
  go: n
  line 11:     for command in ["huh", "go home", "go n"]:
  retu 11:     for command in ["huh", "go home", "go n"]:
  call 22: def match_with_wildcard():
  line 23:     for command in ["huh", "go home", "go n"]:
  line 24:         print(command)
  huh
  line 25:         match command.split():
  line 26:             case ["go", direction] if direction in "nesw":
* line 27:                 match = f"go: {direction}"
  line 28:             case ["go", _]:
* line 29:                 match = "no go"
  line 30:             case x:
  line 31:                 match = f"default: {x}"
  line 32:         print(match)
  default: ['huh']
  line 23:     for command in ["huh", "go home", "go n"]:
  line 24:         print(command)
  go home
  line 25:         match command.split():
  line 26:             case ["go", direction] if direction in "nesw":
  line 28:             case ["go", _]:
  line 29:                 match = "no go"
  line 32:         print(match)
  no go
  line 23:     for command in ["huh", "go home", "go n"]:
  line 24:         print(command)
  go n
  line 25:         match command.split():
  line 26:             case ["go", direction] if direction in "nesw":
  line 27:                 match = f"go: {direction}"
  line 32:         print(match)
  go: n
  line 23:     for command in ["huh", "go home", "go n"]:
  retu 23:     for command in ["huh", "go home", "go n"]:
  call 34: def match_without_wildcard():
  line 35:     match = None
  line 36:     for command in ["huh", "go home", "go n"]:
  line 37:         print(command)
  huh
  line 38:         match command.split():
  line 39:             case ["go", direction] if direction in "nesw":
* line 40:                 match = f"go: {direction}"
  line 41:             case ["go", _]:
* line 42:                 match = "no go"
  line 43:         print(match)
  None
  line 36:     for command in ["huh", "go home", "go n"]:
  line 37:         print(command)
  go home
  line 38:         match command.split():
  line 39:             case ["go", direction] if direction in "nesw":
  line 41:             case ["go", _]:
  line 42:                 match = "no go"
  line 43:         print(match)
  no go
  line 36:     for command in ["huh", "go home", "go n"]:
  line 37:         print(command)
  go n
  line 38:         match command.split():
  line 39:             case ["go", direction] if direction in "nesw":
  line 40:                 match = f"go: {direction}"
  line 43:         print(match)
  go: n
  line 36:     for command in ["huh", "go home", "go n"]:
  retu 36:     for command in ["huh", "go home", "go n"]:
msg397266 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-11 22:10
Thanks, I'll take a closer look at this soon (most likely this coming week).

Out of curiosity, do we have a standard way of writing regression tests for correct line numbers like this? Of the top of my head, it seems like we could either statically compare the dis output (fragile) or actually execute some code with a trace function enabled and check the line hits (tricky).

Neither really seems ideal, so I'm wondering if there's be a better way?
msg397346 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2021-07-12 18:53
Brandt, maybe this regression test from a previous tracing bug might be useful for the test writing: https://github.com/python/cpython/pull/22026/files#diff-8b73bfc55514d8add8904c5f4d1d24b7b644ebfccba8d846085303577aa94dd6
msg397574 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-15 18:47
Thanks, that test framework looks good for this.

My initial hunch (just from looking at this) is that this has to do with how we handle cleanup after failed matches. Our "fail pop" blocks probably have whatever the last line number compiled was (which I think is always the last line of the preceding case block).

The fix *should* be as simple as calling "SET_LOC(c, whatever_the_last_pattern_was)" before compiling these blocks.

I have some new-ish contributors who might want to take this. As far as compiler issues go, this one seems pretty straightforward.
msg397738 - (view) Author: Jack DeVries (jack__d) * Date: 2021-07-18 03:54
@brandtbucher, is anyone working on this yet? I'd like to take a crack at it this week if it's still available!
msg397756 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-18 17:18
Yeah, this is actively being worked on. Thanks for asking.

If anything changes, I’ll let you know and you can pick it up then!
msg398091 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-23 19:59
Two other things we realized while working on this:

- The first occurrence of line 15 in the example output should be marked as incorrectly traced.

- We should emit a NOP to show coverage of "case _" (that fix will be part of the same PR).
msg398145 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-07-24 12:54
The release candidate is soon, so I would recommend to land a fix as soon as possible so it can be tested for some time before is released, if that is possible.
msg398163 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-24 15:25
Yup, I plan on having it in this weekend.
msg398199 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2021-07-25 23:42
New changeset 4214f470f0cb9b6fef9a90758756fbc00ba95b5a by Charles Burkland in branch 'main':
bpo-44600: Fix line numbers for pattern matching cleanup code (GH-27346)
https://github.com/python/cpython/commit/4214f470f0cb9b6fef9a90758756fbc00ba95b5a
msg398200 - (view) Author: miss-islington (miss-islington) Date: 2021-07-26 00:04
New changeset 01601aa7360ae51e74a64dbe957288096bb364fd by Miss Islington (bot) in branch '3.10':
[3.10] bpo-44600: Fix line numbers for pattern matching cleanup code (GH-27346) (GH-27356)
https://github.com/python/cpython/commit/01601aa7360ae51e74a64dbe957288096bb364fd
History
Date User Action Args
2021-07-26 00:04:29miss-islingtonsetmessages: + msg398200
2021-07-25 23:43:03brandtbuchersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-07-25 23:42:43miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25897
2021-07-25 23:42:28brandtbuchersetmessages: + msg398199
2021-07-25 04:42:20chaburklandsetkeywords: + patch
nosy: + chaburkland

pull_requests: + pull_request25887
stage: patch review
2021-07-24 15:25:56brandtbuchersetmessages: + msg398163
2021-07-24 13:37:37pablogsalsetpriority: normal -> release blocker
2021-07-24 12:54:38pablogsalsetnosy: + pablogsal
messages: + msg398145
2021-07-23 19:59:17brandtbuchersetmessages: + msg398091
2021-07-18 17:18:56brandtbuchersetmessages: + msg397756
2021-07-18 03:54:31jack__dsetnosy: + jack__d
messages: + msg397738
2021-07-15 18:47:39brandtbuchersetmessages: + msg397574
2021-07-12 18:53:02ammar2setnosy: + ammar2
messages: + msg397346
2021-07-11 22:10:08brandtbuchersetassignee: brandtbucher
type: behavior
messages: + msg397266
2021-07-11 22:00:10Mark.Shannonsetnosy: + brandtbucher
2021-07-11 21:02:55nedbatcreate