https://github.com/python/cpython/commit/e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423 causes pyqt5 to segfault an accessing an attribute

To reproduce this:


pip install pyqt5
pip install sip
python -c "import PyQt5.QtCore; PyQt5.QtCore.Qt.Key_Control"  # this segfaults

Setting up faulthandler gives:

jupiter@23:54  ➤  python -X faulthandler -c "import PyQt5.QtCore; PyQt5.QtCore.Qt.Key_Control"
Fatal Python error: Segmentation fault

Current thread 0x00007f52111cd740 (most recent call first):
<no Python frame>
Segmentation fault (core dumped)

It is likely that this should also be reported to riverbank as a pyqt/sip bug, but I'm starting here.  I apologize for not having a simpler reproducing case.




jupiter@00:01  ➤ git bisect log
git bisect start
# good: [2d8757758d0d75882fef0fe0e3c74c4756b3e81e] bpo-40286: Remove C implementation of Random.randbytes() (GH-19797)
git bisect good 2d8757758d0d75882fef0fe0e3c74c4756b3e81e
# bad: [d10091aa171250c67a5079abfe26b8b3964ea39a] bpo-40502: Initialize n->n_col_offset (GH-19988)
git bisect bad d10091aa171250c67a5079abfe26b8b3964ea39a
# good: [c3f001461d5794c81cf5f70e08ae5435fe935ceb] bpo-40491: Fix typo in syntax error for numeric literals (GH-19893)
git bisect good c3f001461d5794c81cf5f70e08ae5435fe935ceb
# good: [c21c51235aa8061da6b0593d6f857f42fd92fd8b] bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868)
git bisect good c21c51235aa8061da6b0593d6f857f42fd92fd8b
# good: [c1c7d8ead9eb214a6149a43e31a3213c52448877] bpo-40397: Refactor typing._GenericAlias (GH-19719)
git bisect good c1c7d8ead9eb214a6149a43e31a3213c52448877
# bad: [c068b53a0ca6ebf740d98e422569d2f705e54f93] bpo-38787: Update structures.rst docs (PEP 573) (GH-19980)
git bisect bad c068b53a0ca6ebf740d98e422569d2f705e54f93
# good: [4638c6429575bd6de26b12b2af5df74d6568b553] bpo-40334: Error message for invalid default args in function call (GH-19973)
git bisect good 4638c6429575bd6de26b12b2af5df74d6568b553
# bad: [8963a7f1f84a05412178b56629508b660d38861b] bpo-40545: Export _PyErr_GetTopmostException() function (GH-19978)
git bisect bad 8963a7f1f84a05412178b56629508b660d38861b
# bad: [e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423] bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)
git bisect bad e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423
# first bad commit: [e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423] bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)

Sorry, forgot to add this is on Linux.

I tried to install pyqt5 on Python compiled from source:

make
./python -m venv env
env/bin/python -m pip install sip
env/bin/python -m pip install pyqt5


But I failed to install pyqt5:

vstinner@apu$ env/bin/python -m pip install pyqt5
Collecting pyqt5
/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/msgpack/fallback.py:133: DeprecationWarning: encoding is deprecated, Use raw=False instead.
  unpacker = Unpacker(None, max_buffer_size=len(packed), **kwargs)
  Using cached https://files.pythonhosted.org/packages/4d/81/b9a66a28fb9a7bbeb60e266f06ebc4703e7e42b99e3609bf1b58ddd232b9/PyQt5-5.14.2.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/vstinner/python/master/env/bin/python /home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpzl766kxn
         cwd: /tmp/pip-install-a2jym86c/pyqt5
    Complete output (33 lines):
    Traceback (most recent call last):
      File "/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 64, in prepare_metadata_for_build_wheel
        hook = backend.prepare_metadata_for_build_wheel
    AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
        main()
      File "/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 66, in prepare_metadata_for_build_wheel
        return _get_wheel_metadata_from_wheel(backend, metadata_directory,
      File "/home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 95, in _get_wheel_metadata_from_wheel
        whl_basename = backend.build_wheel(metadata_directory, config_settings)
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/sipbuild/api.py", line 51, in build_wheel
        project = AbstractProject.bootstrap('pep517')
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/sipbuild/abstract_project.py", line 82, in bootstrap
        project.setup(pyproject, tool, tool_description)
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/sipbuild/project.py", line 410, in setup
        self.apply_user_defaults(tool)
      File "project.py", line 62, in apply_user_defaults
        super().apply_user_defaults(tool)
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/pyqtbuild/project.py", line 86, in apply_user_defaults
        super().apply_user_defaults(tool)
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/sipbuild/project.py", line 215, in apply_user_defaults
        self.builder.apply_user_defaults(tool)
      File "/tmp/pip-build-env-w2h9381n/overlay/lib/python3.9/site-packages/pyqtbuild/builder.py", line 67, in apply_user_defaults
        raise PyProjectOptionException('qmake',
    sipbuild.pyproject.PyProjectOptionException
    /home/vstinner/python/master/Lib/tempfile.py:818: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpqj0vy32i'>
      _warnings.warn(warn_message, ResourceWarning)
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/vstinner/python/master/env/bin/python /home/vstinner/python/master/env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpzl766kxn Check the logs for full command output.

Ah, if I install Python, I can install PyQt5. I installed PyQt5:
   python -X faulthandler -c "import PyQt5.QtCore; PyQt5.QtCore.Qt.Key_Control""
doesn't crash.

I also tested the following hello world, it doesn't crash neither.
---
# https://pythonprogramminglanguage.com/pyqt5-hello-world/
import sys
from PyQt5 import QtCore, QtWidgets
from PyQt5.QtWidgets import QMainWindow, QLabel, QGridLayout, QWidget
from PyQt5.QtCore import QSize    

class HelloWindow(QMainWindow):
    def __init__(self):
        QMainWindow.__init__(self)

        self.setMinimumSize(QSize(640, 480))    
        self.setWindowTitle("Hello world - pythonprogramminglanguage.com") 

        centralWidget = QWidget(self)          
        self.setCentralWidget(centralWidget)   

        gridLayout = QGridLayout(self)     
        centralWidget.setLayout(gridLayout)  

        title = QLabel("Hello World from PyQt", self) 
        title.setAlignment(QtCore.Qt.AlignCenter) 
        gridLayout.addWidget(title, 0, 0)

if __name__ == "__main__":
    app = QtWidgets.QApplication(sys.argv)
    mainWin = HelloWindow()
    mainWin.show()
    sys.exit( app.exec_() )
---

I built Python with:

./configure --cache-file=../python-config.cache --with-pydebug CFLAGS=-O0 --prefix=/opt/py39 --with-system-expat --with-system-ffi
make
make install

What is your OS? How did you build Python? Which SIP and PyQt5 versions did you try?

I also with:

   ./configure --prefix=/opt/py39 --with-system-expat --with-system-ffi

I still cannot reproduce the crash. I tested Python at commit 1c2fa781560608aa4be50c748d4b3f403cfa5035. I tested on Fedora 32 (GCC 10.0.1).

$ /opt/py39/bin/python3.9 -m pip freeze
pyflakes==2.2.0
pyperf==2.0.0
PyQt5==5.14.2
PyQt5-sip==12.7.2

$ /opt/py39/bin/python3.9 -VV
Python 3.9.0a6+ (heads/master:1c2fa78156, May 10 2020, 11:37:38) 
[GCC 10.0.1 20200430 (Red Hat 10.0.1-0.14)]

The script I used for the bisect was:

---

TARGET_ENV=bisect_env


rm -r ~/.pybuild/$TARGET_ENV || true
git clean -xfd
./configure --prefix=/home/tcaswell/.pybuild/$TARGET_ENV
make -j 9
make install
~/.pybuild/$TARGET_ENV/bin/python3 -m venv --copies --clear ~/.virtualenvs/$TARGET_ENV

popd

source ~/.virtualenvs/$TARGET_ENV/bin/activate
pip install --upgrade pip
echo $PATH

pushd ../sip    # this was at "tip" in the hg repo
pip install .
pip install pyqt5
popd

python -c "import PyQt5.QtCore; PyQt5.QtCore.Qt.Key_Control"
retVal=$?
if [ $retVal -ne 0 ]; then
    exit 1
fi

---


I'm on a relatively up-to-date Arch 

➤  gcc --version
gcc (Arch Linux 9.3.0-1) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


---

updating pip is a key step (that I thoughtlessly carried over from my normal build process) because that lets it see manylinux2014 wheels and does not try to build pyqt5 from source.

python3.9 -m pip freeze
packaging==20.3
pyparsing==2.4.7
PyQt5==5.14.2
PyQt5-sip==12.7.2
sip==5.2.0
six==1.14.0
toml==0.10.0

➤  python3.9 -VV
Python 3.9.0a6+ (heads/master-7-g1c2fa78156:1c2fa78156, May 10 2020, 11:11:03) 
[GCC 9.3.0]


----

I'm also having issues getting pyqt5 to compile from the tarball, what do you mean by "Ah, if I install Python, I can install PyQt5. I installed PyQt5:"?

I think I have figured out the problem.  I had a locally built and cached wheel of PyQt5-sip from before PEP573 went in.  If that wheel is used for later commits I get the segfault, if I rebuilt the wheel from source it works.

I am not sure if this is an expected sharp edge (wheels are be expected to be good between commits prior to a release) or not.


-----


Below is a script that will build everything (sip, pyqt5, pyqt5-sip) from source and install them into a virtual env.

----
#! /usr/bin/bash
set -e
set -o xtrace



rm -r ~/.pybuild/py39 || true
pushd cpython
git clean -xfd
./configure --prefix=/home/tcaswell/.pybuild/py39
make -j 9
make install
popd

~/.pybuild/py39/bin/python3 -m venv --copies --clear ~/.virtualenvs/py39



source ~/.virtualenvs/py39/bin/activate
pip install --upgrade pip
echo $PATH

pushd PyQt5
pip install sip --no-binary sip
rm -rf PyQt5-5.14.2
tar -xzf PyQt5-5.14.2.tar.gz

pushd PyQt5-5.14.2
python configure.py --confirm-license
# patch the one palce that does not respect the venv settings
sed -i 's|$(INSTALL_ROOT)/usr|$(INSTALL_ROOT)/$(HOME)/.virtualenvs/py39/usr|g' Makefile
make -j 9
make install
popd
popd
pip install PyQt5-sip --no-binary PyQt5-sip

python -c "import PyQt5.QtCore; PyQt5.QtCore.Qt.Key_Control"


----

> I think I have figured out the problem.  I had a locally built and cached wheel of PyQt5-sip from before PEP573 went in.  If that wheel is used for later commits I get the segfault, if I rebuilt the wheel from source it works.

Hum, I'm not sure that I understand well. Is there a bug in Python or not?

It seems when in my tests, pip of Python 3.9 installed wheel package marked as "python38". I understand that the ABI didn't change and we are all good.

I suggest to close the issue.

The path is

 - on a commit prior to e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423 install pyqt-sip.  pip will build a wheel for you called PyQt5_sip-12.7.2-cp39-cp39-linux_x86_64.whl
 - on a commit after e1becf46b4e3ba6d7d32ebf4bbd3e0804766a423 if you do `pip install pyqt-sip` pip will discover the wheel from the previous install in it's cache and use it, but the ABI has changed (?) which leads to a segfault.  If you do `pip install pyqt5-sip --no-binary pyqt5-sip` or clear the cache then things work correctly

I suspect that this also means that wheels made with the early alphas will not work future 3.9 (pre-)releases, however I am not sure if that is a problem or not (given that it is all pre-releases).

I am going to close this as I think if there is a bug, it would be better addressed in the packaging space.

The stable ABI should not change between Python 3.8 and 3.9.

In practice, it seems like something changed. But without any gdb traceback, I cannot tell what.

I suggest to try again when beta1 will be released. The ABI should be way more stable after beta1.

So, the failure is expected. Python's ABI can change until the 3.9.0 final release, so wheels built for different commits can be incompatible.
This applies to alphas/betas as well, as you say.

There is the PEP 384 stable ABI, which is much stricter (and more limited), but sip doesn't use it -- its wheels are specific to cp36/cp37/cp38: https://pypi.org/project/sip/#files

Thanks for testing, though! Perhaps we need to communicate better that for the alphas/betas, everything needs to be rebuilt.

That seems reasonable.

To be pedantic, it is pyqt5-sip (not sip) that was the source of the problem.

I am going to open an issue with pip to disable caching locally built wheels for pre-released versions of Python.