classification
Title: pythoninfo collect_gdb() blows up when gdb fails to run
Type: Stage: resolved
Components: Tests Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: hroncok, miss-islington, vstinner
Priority: normal Keywords: patch

Created on 2020-04-29 11:36 by hroncok, last changed 2020-06-11 13:23 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 19792 merged vstinner, 2020-04-29 14:07
PR 19795 merged miss-islington, 2020-04-29 15:12
PR 19796 merged miss-islington, 2020-04-29 15:13
Messages (8)
msg367641 - (view) Author: Miro Hrončok (hroncok) * Date: 2020-04-29 11:36
We had this weird traceback when running pythoninfo in Fedora build with Python 3.9.0a6:

+ /builddir/build/BUILD/Python-3.9.0a6/build/optimized/python -m test.pythoninfo
ERROR: collect_gdb() failed
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/pythoninfo.py", line 761, in collect_info
    collect_func(info_add)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/pythoninfo.py", line 383, in collect_gdb
    version = version.splitlines()[0]
IndexError: list index out of range


I have debugged the problem and it is:


>>> subprocess.run(["gdb", "-nx", "--version"])
CompletedProcess(args=['gdb', '-nx', '--version'], returncode=-11)

There is a segfault. Possibly because gdb was linked to libpython from 3.9.0a5 and we run it trough subprocess from 3.9.0a6.


The code in pythoninfo is:

    try:
        proc = subprocess.Popen(["gdb", "-nx", "--version"],
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE,
                                universal_newlines=True)
        version = proc.communicate()[0]
    except OSError:
        return

That means it is designed to ignore errors. But it only ignores some errors. Should it only attempt to parse the version when proc.returncode is 0?

I don't know yet if the tests will fail as well, maybe the problem will be bigger.
msg367647 - (view) Author: Miro Hrončok (hroncok) * Date: 2020-04-29 14:09
BTW The test gdb also crashes in the same way:

test test_gdb crashed -- Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/test_gdb.py", line 41, in <module>
    gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version()
  File "/builddir/build/BUILD/Python-3.9.0a6/Lib/test/test_gdb.py", line 38, in get_gdb_version
    raise Exception("unable to parse GDB version: %r" % version)
Exception: unable to parse GDB version: ''

1 test failed again:
    test_gdb
msg367654 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 15:11
New changeset ec9bea4a3766bd815148a27f61eb24e7dd459ac7 by Victor Stinner in branch 'master':
bpo-40436: Fix code parsing gdb version (GH-19792)
https://github.com/python/cpython/commit/ec9bea4a3766bd815148a27f61eb24e7dd459ac7
msg367657 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 15:16
I fixed test.pythoninfo to ignore gdb output if the command failed. I also enhanced test_gdb error message: display stdout, stderr and the exit code. Backport to 3.8 and 3.7 will be merged as soon as the CI pass.
msg367661 - (view) Author: miss-islington (miss-islington) Date: 2020-04-29 15:30
New changeset d9e904919197a22b95946f11ba5f24b796088c06 by Miss Islington (bot) in branch '3.8':
bpo-40436: Fix code parsing gdb version (GH-19792)
https://github.com/python/cpython/commit/d9e904919197a22b95946f11ba5f24b796088c06
msg367662 - (view) Author: miss-islington (miss-islington) Date: 2020-04-29 15:30
New changeset beba1a808000d5fc445cb28eab96bdb4cdb7c959 by Miss Islington (bot) in branch '3.7':
bpo-40436: Fix code parsing gdb version (GH-19792)
https://github.com/python/cpython/commit/beba1a808000d5fc445cb28eab96bdb4cdb7c959
msg367663 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-04-29 15:32
Thanks Miro for the bug report, it's now fixed ;-)

In PR 19792, Miro proposed to skip test_gdb is gdb is available but exit with non-zero exit code. I chose to raise a hard exception instead, to notify CI owners that something is wrong. I prefer to not make gdb error "silent". Otherwise, we may miss that python-gdb.py is no longer tested on a CI.
msg371276 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-11 13:23
I marked bpo-29685 "test_gdb failed" as duplicate of this issue. Extract:

(...)
  File "/home/studio/Python-3.6.0/Lib/test/test_gdb.py", line 46, in <module>
    gdb_version, gdb_major_version, gdb_minor_version = get_gdb_version()
  File "/home/studio/Python-3.6.0/Lib/test/test_gdb.py", line 43, in get_gdb_version
    raise Exception("unable to parse GDB version: %r" % version)
Exception: unable to parse GDB version: ''
History
Date User Action Args
2020-06-11 13:23:37vstinnersetmessages: + msg371276
2020-06-11 13:22:55vstinnerlinkissue29685 superseder
2020-04-29 15:32:54vstinnersetmessages: + msg367663
2020-04-29 15:30:57miss-islingtonsetmessages: + msg367662
2020-04-29 15:30:09miss-islingtonsetmessages: + msg367661
2020-04-29 15:16:02vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg367657

stage: patch review -> resolved
2020-04-29 15:13:48miss-islingtonsetpull_requests: + pull_request19117
2020-04-29 15:12:21miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19116
2020-04-29 15:11:51vstinnersetmessages: + msg367654
2020-04-29 14:09:28hroncoksetmessages: + msg367647
2020-04-29 14:07:57vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request19114
2020-04-29 11:36:45hroncokcreate