I've observed that pytest becomes segmentation fault on python 3.9.0 with attached case.

I've tested the case with several python versions;
python 3.9.0a2 - good
python 3.9.0a3, 3.9.0-final - bad
python 3.10.0a1 - good

• OS: Mint Linux 20, Linux kernel 5.8.14
• Attachments:
  • test_main.py - pytest test case to reproduce
  • py_stacktrace.txt - pytest result which become segmentation fault
  • gdb_backtrace.txt - gdb backtrace

So I've bisected and a result is as follows;

9af0e47 is the first bad commit
commit 9af0e47
Author: Mark Shannon <mark@hotpy.org>
Date: Tue Jan 14 10:12:45 2020 +0000

bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)

Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.

Doc/library/dis.rst | 21 +
Include/opcode.h | 8 +-
Lib/importlib/_bootstrap_external.py | 3 +-
Lib/opcode.py | 7 +-
Lib/test/test_dis.py | 141 +-
Lib/test/test_peepholer.py | 12 +-
Lib/test/test_positional_only_arg.py | 6 +-
.../2019-12-30-10-53-59.bpo-39156.veT-CB.rst | 9 +
PC/launcher.c | 3 +-
Python/ceval.c | 137 +-
Python/compile.c | 71 +-
Python/importlib.h | 2922 +++++++------
Python/importlib_external.h | 4560 ++++++++++----------
Python/importlib_zipimport.h | 1831 ++++----
Python/opcode_targets.h | 6 +-
Python/peephole.c | 6 +-
Tools/scripts/generate_opcode_h.py | 5 -
17 files changed, 4901 insertions(+), 4847 deletions(-)
create mode 100644 Misc/NEWS.d/next/Core and Builtins/2019-12-30-10-53-59.bpo-39156.veT-CB.rst

Here is a result of running pytest on python 3.9.0 with gdb.

==================================================== test session starts ====================================================
platform linux -- Python 3.9.0, pytest-4.6.9, py-1.8.1, pluggy-0.13.0
rootdir: /home/miurahr/Projects/cpython
collected 2 items

test_main.py .
Program received signal SIGSEGV, Segmentation fault.
PyException_GetContext (self=self@entry=<traceback at remote 0x7ffff62b47c0>) at ../Objects/exceptions.c:351
warning: Source file is more recent than executable.
351 Py_XINCREF(context);

FYI: A following commit fixes the issue in 3.10 development branch.

6e8128f ("bpo-41323: Perform 'peephole' optimizations directly on the CFG. (GH-21517)", 2020-07-30)

A test code does not always reproduce the issue. Please try it in several times.

It seems to be happened when multiple threads try execute a same function which produces an exception, and both callers try to catch the exception at the same time.

The test example has no reference to pytest in it.

How are you running it?
Can you produce a segmentation fault when run without pytest?

I confirmed the issue. The simple version of the reproducer is:

def callee():
    raise Exception

def caller():
    try:
        callee()
    except Exception or Exception:
        pass

caller()

I can see assertion failure consisntently, when I use python --with-pydebug.

It seems peephole bug. Without peephole:

  7     >>   12 DUP_TOP
             14 LOAD_GLOBAL              1 (Exception)
             16 JUMP_IF_TRUE_OR_POP     20
             18 LOAD_GLOBAL              1 (Exception)
        >>   20 JUMP_IF_NOT_EXC_MATCH    32
             22 POP_TOP
             24 POP_TOP
             26 POP_TOP

  8          28 POP_EXCEPT
             30 JUMP_FORWARD             2 (to 34)
        >>   32 RERAISE
        >>   34 LOAD_CONST               0 (None)
             36 RETURN_VALUE

And with peephole:

  7     >>   12 DUP_TOP
             14 LOAD_GLOBAL              1 (Exception)
             16 POP_JUMP_IF_TRUE        22
             18 LOAD_GLOBAL              1 (Exception)
             20 JUMP_IF_NOT_EXC_MATCH    32
        >>   22 POP_TOP
             24 POP_TOP
             26 POP_TOP

  8          28 POP_EXCEPT
             30 JUMP_FORWARD             2 (to 34)
        >>   32 RERAISE
        >>   34 LOAD_CONST               0 (None)
             36 RETURN_VALUE

JUMP_IF_TRUE_OR_POP is converted into POP_JUMP_IF_TRUE. Exception is popped although JUMP_IF_NOT_EXC_MATCH needs it.

I confirmed fix42057.patch fix the assertion failure. But I don't know where and how to write test yet.

New changeset 07a44d9 by Inada Naoki in branch '3.9':
bpo-42057: Fix peephole optimizer (GH-22802)
07a44d9

New changeset 8f6787d by Inada Naoki in branch '3.9':
bpo-42057: Add a test case (GH-22878)
8f6787d

Thank you for reporting with reproducer.

New changeset b52432c by Mark Shannon in branch 'master':
bpo-42057: Add regression test to master. (GH-22893)
b52432c