classification
Title: Segfault when using trace functions in 3.11a3
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, Quentin.Pradet, alex, reaperhulk
Priority: normal Keywords:

Created on 2021-12-23 03:07 by reaperhulk, last changed 2022-01-18 06:42 by Quentin.Pradet.

Messages (4)
msg409061 - (view) Author: Paul Kehrer (reaperhulk) Date: 2021-12-23 03:07
In Python 3.11a3 on Linux/x86_64 (failed to replicate on macOS, not attempted on Windows) the interpreter non-deterministically segfaults when running some code under coverage. This did not occur under 3.11a2. Looking at the backtrace from a core dump I see:

#0  _PyFrame_FastToLocalsWithError (frame=0x7fedf9e1f608) at Objects/frameobject.c:903
#1  0x00007fedfa15f593 in call_trampoline (tstate=0x55b767a44080, callback=0x7fedf8bbd9c0, 

This is the trace received if I use pure Python coverage (sys.settrace) while I get one inside coverage's ctracer if I use the native library. However, at the moment I don't believe the bug resides within coverage.

Since stack frame optimization has been a focus in 3.11 could something have changed that is causing issues with sys.settrace/PyEval_SetTrace?

I haven't managed to reduce this test case much but here's a somewhat messy dockerfile that can demonstrate it:

FROM ubuntu:focal
RUN apt-get update && apt-get install -y build-essential git cargo libffi-dev libssl-dev libsqlite3-dev zlib1g-dev curl
RUN curl -OL https://www.python.org/ftp/python/3.11.0/Python-3.11.0a3.tgz && \
    tar zxf Python-3.11* && \
    cd Python-3.11* && \
    ./configure --prefix=/opt && \
    make -j4 && make install
RUN /opt/bin/pip3 install tox && git clone https://github.com/pyca/cryptography
RUN cd cryptography && /opt/bin/tox -e py311
msg410385 - (view) Author: Quentin Pradet (Quentin.Pradet) * Date: 2022-01-12 05:46
The ABI between alpha 2 and alpha 3 changed, see https://bugs.python.org/issue46320. If pip built (say) cffi in CI for alpha2 and put it in the cache, then it reused that wheel for alpha3 and the segfault is expected.
msg410817 - (view) Author: Paul Kehrer (reaperhulk) Date: 2022-01-17 18:33
Changes in ABI don't seem to be the likely culprit since the Dockerfile provided can demonstrate this bug and has no caching that would result in obtaining alpha2-based binaries.
msg410842 - (view) Author: Quentin Pradet (Quentin.Pradet) * Date: 2022-01-18 06:42
Thanks, you're right, I ran `git bisect` on Linux and the commit that causes the issue is https://github.com/python/cpython/commit/135cabd328504e1648d17242b42b675cdbd0193b ("Copy free variables in bytecode to allow calls to inner functions to be specialized") from https://bugs.python.org/issue44525
History
Date User Action Args
2022-01-18 06:42:14Quentin.Pradetsetmessages: + msg410842
2022-01-17 18:33:06reaperhulksetmessages: + msg410817
2022-01-12 05:46:47Quentin.Pradetsetnosy: + Quentin.Pradet
messages: + msg410385
2021-12-23 03:11:18alexsetnosy: + alex, Mark.Shannon
components: + Interpreter Core
2021-12-23 03:07:56reaperhulksettitle: Segfault -> Segfault when using trace functions in 3.11a3
2021-12-23 03:07:26reaperhulkcreate