classification
Title: test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: koobs, miss-islington, vstinner, zitterbewegung
Priority: normal Keywords: patch

Created on 2019-03-04 14:23 by vstinner, last changed 2020-03-11 02:11 by koobs. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18538 closed Ananthakrishnan, 2020-02-17 18:25
PR 18873 merged vstinner, 2020-03-09 12:31
PR 18881 merged miss-islington, 2020-03-09 18:35
PR 18882 merged miss-islington, 2020-03-09 18:35
Messages (10)
msg337120 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-04 14:23
On my FreeBSD VM, _POSIX_THREADS is not defined:

sys.thread_info: sys.thread_info(name='pthread', lock='semaphore', version=None)

test_gdb fails with:

Verify that "py-bt" indicates threads that are waiting for the GIL ... FAIL

======================================================================
FAIL: test_threads (test.test_gdb.PyBtTests)
Verify that "py-bt" indicates threads that are waiting for the GIL
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_gdb.py", line 826, in test_threads
    self.assertIn('Waiting for the GIL', gdb_output)
AssertionError: 'Waiting for the GIL' not found in 'Breakpoint 1 at 0x49e5d0: file Python/bltinmodule.c, line 1203.\n[New LWP 100742 of process 68315]\n[New LWP 100749 of process 68315]\n[New LWP 100751 of process 68315]\n[New LWP 100766 of process 68315]\n\nThread 1 hit Breakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1203\n1203\t    return PyLong_FromVoidPtr(v);\n\nThread 5 (LWP 100766 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 4 (LWP 100751 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 3 (LWP 100749 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 2 (LWP 100742 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 1 (LWP 100559 of process 68315):\nTraceback (most recent call first):\n  <built-in method id of module object at remote 0x800bccde8>\n  File "<string>", line 18, in <module>\n'


=> 'Waiting for the GIL' cannot be found in the output, because python-gdb.py failed to detect that a threading is waiting for the GIL.


The problem can be found in Tools/gdb/libpython.py:

    def is_waiting_for_gil(self):
        '''Is this frame waiting on the GIL?'''
        # This assumes the _POSIX_THREADS version of Python/ceval_gil.h:
        name = self._gdbframe.name()
        if name:
            return 'pthread_cond_timedwait' in name

pthread_cond_timedwait() is too close to POSIX threads. We can make this function more portable by checking for 'take_gil' function instead.
msg337643 - (view) Author: Joshua Jay Herman (zitterbewegung) * Date: 2019-03-11 01:59
Hi, I would like to try to solve this issue. Does this occur on the latest version of FreeBSD?
msg337687 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-11 17:07
I am running FreeBSD 12. I don't recall the minor version (12.1 maybe?).
msg337712 - (view) Author: Joshua Jay Herman (zitterbewegung) * Date: 2019-03-12 02:52
I was able to reproduce this on FreeBSD 12.0.
msg338050 - (view) Author: Joshua Jay Herman (zitterbewegung) * Date: 2019-03-16 02:31
So I changed 'pthread_cond_timedwait' to 'take_gil' and this didn't seem to have an effect on the unit test.

I installed GDB 8.2 is that the same version you are using?
msg363723 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-09 12:32
> So I changed 'pthread_cond_timedwait' to 'take_gil' and this didn't seem to have an effect on the unit test.

Maybe you forgot to run "make" to copy Tools/gdb/libpython.py as python-gdb.py.

Anyway, I wrote PR 18873 and I tested manually that it does fix test_gdb on FreeBSD. I tested "GNU gdb (GDB) 8.3.1 [GDB v8.3.1 for FreeBSD]" on FreeBSD 12.1-RELEASE-p2.
msg363759 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-09 18:35
New changeset 6d0ee60740f2862a878f009671b1aaa75aeb0c2a by Victor Stinner in branch 'master':
bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
https://github.com/python/cpython/commit/6d0ee60740f2862a878f009671b1aaa75aeb0c2a
msg363761 - (view) Author: miss-islington (miss-islington) Date: 2020-03-09 18:51
New changeset 1695836123609a8ae97f2cfbe180a028dcd650a3 by Miss Islington (bot) in branch '3.7':
bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
https://github.com/python/cpython/commit/1695836123609a8ae97f2cfbe180a028dcd650a3
msg363762 - (view) Author: miss-islington (miss-islington) Date: 2020-03-09 18:52
New changeset 5854d451cb35ac38bc07b95afeb077e8a35d5c7d by Miss Islington (bot) in branch '3.8':
bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
https://github.com/python/cpython/commit/5854d451cb35ac38bc07b95afeb077e8a35d5c7d
msg363876 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2020-03-11 02:11
For our future reference/selves, GDB (currently 9.1 w/ python bindings) is now installed on the two FreeBSD buildbot workers
History
Date User Action Args
2020-03-11 02:11:21koobssetmessages: + msg363876
2020-03-09 19:14:38vstinnersetkeywords: - easy
status: open -> closed
stage: patch review -> resolved
resolution: fixed
title: [EASY] test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD -> test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD
2020-03-09 18:52:31miss-islingtonsetmessages: + msg363762
2020-03-09 18:51:47miss-islingtonsetmessages: + msg363761
2020-03-09 18:35:50miss-islingtonsetpull_requests: + pull_request18239
2020-03-09 18:35:41miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request18238
2020-03-09 18:35:33vstinnersetmessages: + msg363759
2020-03-09 12:32:55vstinnersetmessages: + msg363723
2020-03-09 12:31:45vstinnersetpull_requests: + pull_request18231
2020-02-17 18:25:16Ananthakrishnansetkeywords: + patch
stage: patch review
pull_requests: + pull_request17915
2019-03-16 02:31:09zitterbewegungsetmessages: + msg338050
2019-03-12 02:57:22koobssetnosy: + koobs
2019-03-12 02:52:43zitterbewegungsetmessages: + msg337712
2019-03-11 17:07:03vstinnersetmessages: + msg337687
2019-03-11 01:59:55zitterbewegungsetnosy: + zitterbewegung
messages: + msg337643
2019-03-04 14:25:40vstinnersetkeywords: + easy
title: test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD -> [EASY] test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD
2019-03-04 14:23:48vstinnercreate