Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD #80365

Closed
vstinner opened this issue Mar 4, 2019 · 10 comments
Closed

test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD #80365

vstinner opened this issue Mar 4, 2019 · 10 comments
Labels
3.8 only security fixes stdlib Python modules in the Lib dir

Comments

@vstinner
Copy link
Member

vstinner commented Mar 4, 2019

BPO 36184
Nosy @vstinner, @koobs, @zitterbewegung, @miss-islington
PRs
  • bpo-36184:Made 'is_waiting_for_gil' function more portable. #18538
  • bpo-36184: Port python-gdb.py to FreeBSD #18873
  • [3.8] bpo-36184: Port python-gdb.py to FreeBSD (GH-18873) #18881
  • [3.7] bpo-36184: Port python-gdb.py to FreeBSD (GH-18873) #18882
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-03-09.19:14:38.122>
created_at = <Date 2019-03-04.14:23:48.586>
labels = ['3.8', 'library']
title = 'test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD'
updated_at = <Date 2020-03-11.02:11:21.575>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-03-11.02:11:21.575>
actor = 'koobs'
assignee = 'none'
closed = True
closed_date = <Date 2020-03-09.19:14:38.122>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2019-03-04.14:23:48.586>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 36184
keywords = ['patch']
message_count = 10.0
messages = ['337120', '337643', '337687', '337712', '338050', '363723', '363759', '363761', '363762', '363876']
nosy_count = 4.0
nosy_names = ['vstinner', 'koobs', 'zitterbewegung', 'miss-islington']
pr_nums = ['18538', '18873', '18881', '18882']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36184'
versions = ['Python 3.8']
    @vstinner
    Copy link
    Member Author

    vstinner commented Mar 4, 2019

    On my FreeBSD VM, _POSIX_THREADS is not defined:

    sys.thread_info: sys.thread_info(name='pthread', lock='semaphore', version=None)

    test_gdb fails with:

    Verify that "py-bt" indicates threads that are waiting for the GIL ... FAIL

    ======================================================================
    FAIL: test_threads (test.test_gdb.PyBtTests)
    Verify that "py-bt" indicates threads that are waiting for the GIL
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/usr/home/vstinner/prog/python/master/Lib/test/test_gdb.py", line 826, in test_threads
    self.assertIn('Waiting for the GIL', gdb_output)
AssertionError: 'Waiting for the GIL' not found in 'Breakpoint 1 at 0x49e5d0: file Python/bltinmodule.c, line 1203.\n[New LWP 100742 of process 68315]\n[New LWP 100749 of process 68315]\n[New LWP 100751 of process 68315]\n[New LWP 100766 of process 68315]\n\nThread 1 hit Breakpoint 1, builtin_id (self=, v=42) at Python/bltinmodule.c:1203\n1203\t    return PyLong_FromVoidPtr(v);\n\nThread 5 (LWP 100766 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 4 (LWP 100751 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 3 (LWP 100749 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 2 (LWP 100742 of process 68315):\nTraceback (most recent call first):\n  File "<string>", line 10, in run\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 917, in _bootstrap_inner\n    self.run()\n  File "/usr/home/vstinner/prog/python/master/Lib/threading.py", line 885, in _bootstrap\n    self._bootstrap_inner()\n\nThread 1 (LWP 100559 of process 68315):\nTraceback (most recent call first):\n  <built-in method id of module object at remote 0x800bccde8>\n  File "<string>", line 18, in <module>\n'

    => 'Waiting for the GIL' cannot be found in the output, because python-gdb.py failed to detect that a threading is waiting for the GIL.

    The problem can be found in Tools/gdb/libpython.py:

        def is_waiting_for_gil(self):
        '''Is this frame waiting on the GIL?'''
        # This assumes the _POSIX_THREADS version of Python/ceval_gil.h:
        name = self._gdbframe.name()
        if name:
            return 'pthread_cond_timedwait' in name

    pthread_cond_timedwait() is too close to POSIX threads. We can make this function more portable by checking for 'take_gil' function instead.

    @vstinner vstinner added 3.8 only security fixes stdlib Python modules in the Lib dir easy labels Mar 4, 2019
    @vstinner vstinner changed the title test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD [EASY] test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD Mar 4, 2019
    @zitterbewegung
    Copy link
    Mannequin

    zitterbewegung mannequin commented Mar 11, 2019

    Hi, I would like to try to solve this issue. Does this occur on the latest version of FreeBSD?

    @vstinner
    Copy link
    Member Author

    I am running FreeBSD 12. I don't recall the minor version (12.1 maybe?).

    @zitterbewegung
    Copy link
    Mannequin

    zitterbewegung mannequin commented Mar 12, 2019

    I was able to reproduce this on FreeBSD 12.0.

    @zitterbewegung
    Copy link
    Mannequin

    zitterbewegung mannequin commented Mar 16, 2019

    So I changed 'pthread_cond_timedwait' to 'take_gil' and this didn't seem to have an effect on the unit test.

    I installed GDB 8.2 is that the same version you are using?

    @vstinner
    Copy link
    Member Author

    vstinner commented Mar 9, 2020

    So I changed 'pthread_cond_timedwait' to 'take_gil' and this didn't seem to have an effect on the unit test.

    Maybe you forgot to run "make" to copy Tools/gdb/libpython.py as python-gdb.py.

    Anyway, I wrote PR 18873 and I tested manually that it does fix test_gdb on FreeBSD. I tested "GNU gdb (GDB) 8.3.1 [GDB v8.3.1 for FreeBSD]" on FreeBSD 12.1-RELEASE-p2.

    @vstinner
    Copy link
    Member Author

    vstinner commented Mar 9, 2020

    New changeset 6d0ee60 by Victor Stinner in branch 'master':
    bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
    6d0ee60

    @miss-islington
    Copy link
    Contributor

    New changeset 1695836 by Miss Islington (bot) in branch '3.7':
    bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
    1695836

    @miss-islington
    Copy link
    Contributor

    New changeset 5854d45 by Miss Islington (bot) in branch '3.8':
    bpo-36184: Port python-gdb.py to FreeBSD (GH-18873)
    5854d45

    @vstinner vstinner removed the easy label Mar 9, 2020
    @vstinner vstinner closed this as completed Mar 9, 2020
    @vstinner vstinner changed the title [EASY] test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD test_gdb.test_threads() is specific to _POSIX_THREADS, fail on FreeBSD Mar 9, 2020
    @koobs
    Copy link

    koobs commented Mar 11, 2020

    For our future reference/selves, GDB (currently 9.1 w/ python bindings) is now installed on the two FreeBSD buildbot workers

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @koobs @miss-islington