This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: readline + GTK + Pytest Seg Fault with Python 3.7 and 3.10 on CI
Type: crash Stage: resolved
Components: Library (Lib) Versions: Python 3.10, Python 3.7
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: danyeaw, iritkatriel
Priority: normal Keywords:

Created on 2021-11-28 03:53 by danyeaw, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg407184 - (view) Author: Dan Yeaw (danyeaw) Date: 2021-11-28 03:53
When running pytest --doctest-modules, I am getting seg faults on the GitHub Actions CI when running doctests covering module docstrings.

runner@fv-az177-300:~/work/gaphor/gaphor$ source .venv/bin/activate
(.venv) runner@fv-az177-300:~/work/gaphor/gaphor$ xvfb-run gdb python

(gdb) run -m pytest
Starting program: /home/runner/work/gaphor/gaphor/.venv/bin/python -m pytest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffecd32700 (LWP 22846)]
[New Thread 0x7fffebd31700 (LWP 22847)]
[New Thread 0x7fffead30700 (LWP 22848)]
[New Thread 0x7fffe9d2f700 (LWP 22849)]
[New Thread 0x7fffdbfff700 (LWP 22850)]
[New Thread 0x7fffdaffe700 (LWP 22851)]
[New Thread 0x7fffd9ffd700 (LWP 22852)]
[New Thread 0x7fffcffff700 (LWP 22853)]
[Detaching after fork from child process 22854]
============================= test session starts ==============================
platform linux -- Python 3.10.0, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/runner/work/gaphor/gaphor, configfile: pyproject.toml, testpaths: gaphor, tests, docs
plugins: mock-3.6.1, cov-3.0.0
collected 1135 items                                                           

gaphor/action.py 
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0x20) at malloc.c:3102
3102    malloc.c: No such file or directory.
(gdb) source python-gdb.py
(gdb) py-bt
Traceback (most recent call first):
  <built-in method create_dynamic of module object at remote 0x7ffff78ebcb0>
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/pdb.py", line 157, in __init__
    import readline
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 364, in __init__
    pdb.Pdb.__init__(self, stdout=out, nosigint=True)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 1481, in run
    self.debugger = _OutputRedirectingPdb(save_stdout)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/doctest.py", line 1856, in run
    r = DocTestRunner.run(self, test, compileflags, out, False)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/doctest.py", line 287, in runtest
    self.runner.run(self.dtest, out=failures)  # type: ignore[arg-type]
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 162, in pytest_runtest_call
    item.runtest()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 255, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 254, in call_runtest_hook
    return CallInfo.from_call(
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 215, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 126, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 348, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 323, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/runner/work/gaphor/gaphor/.venv/lib/python3.10/site-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  <built-in method exec of module object at remote 0x7ffff78d3d10>
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/hostedtoolcache/Python/3.10.0/x64/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,

action.py is part of the Gaphor project here: https://github.com/gaphor/gaphor

The doctest that it is seg faulting on is:

    >>> class A:
    ...     @action(name="my_action", label="my action")
    ...     def myaction(self):
    ...         print("action called")
    >>> a = A()
    >>> a.myaction()
    action called
    >>> is_action(a.myaction)
    True
    >>> for method in dir(A):
    ...     if is_action(getattr(A, method, None)):
    ...         print(method)
    myaction
    >>> A.myaction.__action__.name
    'my_action'
    >>> A.myaction.__action__.label
    'my action'
    """

Running the doctests works fine manually:
dan@localhost:~/Projects/gaphor> python -m doctest -v gaphor/action.py
Trying:
    class A:
        @action(name="my_action", label="my action")
        def myaction(self):
            print("action called")
Expecting nothing
ok
Trying:
    a = A()
Expecting nothing
ok
Trying:
    a.myaction()
Expecting:
    action called
ok
Trying:
    is_action(a.myaction)
Expecting:
    True
ok
Trying:
    for method in dir(A):
        if is_action(getattr(A, method, None)):
            print(method)
Expecting:
    myaction
ok
Trying:
    A.myaction.__action__.name
Expecting:
    'my_action'
ok
Trying:
    A.myaction.__action__.label
Expecting:
    'my action'
ok
5 items had no tests:
    action
    action.action.__call__
    action.action.__init__
    action.action.detailed_name
    action.is_action
1 items passed all tests:
   7 tests in action.action
7 tests in 6 items.
7 passed and 0 failed.
Test passed.

If I disable this doctests, the other doctests will seg fault as well. Using xdoctest instead of doctest also fixes the seg fault.

I compiled Python 3.10.0 with debug enabled on the GitHub Actions runner, so I can provide the full backtrace if that is helpful.
msg407207 - (view) Author: Dan Yeaw (danyeaw) Date: 2021-11-28 13:29
I continued to look in to this and updated the title of the issue, I don't think this is caused by doctest. I also updated a minimum example here:

https://github.com/danyeaw/plugin-console-test

If using pytest and GTK, any module loaded that makes use of readline causes a seg fault on CI. For example rlcompleter and doctest cause the crash, and even just importing readline.

This is reproducible for Python 3.10 and Python 3.7, but not 3.8 or 3.9.
msg410827 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-17 22:39
There are a number of third party components involved in your bug. If you can't narrow it down to something that plausibly looks like a cpython bug, it's unlikely that a core dev will investigate it.
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 90071
2022-01-29 13:06:14iritkatrielsetstatus: pending -> closed
stage: resolved
2022-01-17 22:39:21iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg410827

resolution: third party
2021-11-28 13:29:09danyeawsetversions: + Python 3.7
messages: + msg407207
title: Doctest Seg Fault with Python 3.10 on CI -> readline + GTK + Pytest Seg Fault with Python 3.7 and 3.10 on CI
2021-11-28 03:53:41danyeawcreate