Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate thread debugging PYTHONTHREADDEBUG=1 #88750

Closed
vstinner opened this issue Jul 8, 2021 · 5 comments · Fixed by #92509
Closed

Deprecate thread debugging PYTHONTHREADDEBUG=1 #88750

vstinner opened this issue Jul 8, 2021 · 5 comments · Fixed by #92509
Labels
3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

vstinner commented Jul 8, 2021

BPO 44584
Nosy @vstinner, @miss-islington
PRs
  • bpo-44584: Deprecate PYTHONTHREADDEBUG env var #27065
  • [3.10] bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065) #27624
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-08-06.12:14:30.875>
    created_at = <Date 2021-07-08.09:23:08.980>
    labels = ['interpreter-core', '3.10', '3.11']
    title = 'Deprecate thread debugging PYTHONTHREADDEBUG=1'
    updated_at = <Date 2021-08-06.12:14:30.874>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-08-06.12:14:30.874>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-08-06.12:14:30.875>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2021-07-08.09:23:08.980>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44584
    keywords = ['patch']
    message_count = 5.0
    messages = ['397130', '397131', '399071', '399074', '399077']
    nosy_count = 2.0
    nosy_names = ['vstinner', 'miss-islington']
    pr_nums = ['27065', '27624']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue44584'
    versions = ['Python 3.10', 'Python 3.11']

    @vstinner
    Copy link
    Member Author

    vstinner commented Jul 8, 2021

    Copy of my email to python-dev:
    https://mail.python.org/archives/list/python-dev@python.org/thread/NMLGCDRUKLZSTK4UICJTKR54WRXU2ZGJ/

    Hi,

    Does anyone use threading debug PYTHONTHREADDEBUG=1 env var on a
    Python debug build? If not, can I just remove it?

    --

    To fix a race condition at thread exit on Linux using the glibc, I
    removed calls to pthread_exit() (PyThread_exit_thread()) in the
    _thread module:

    https://bugs.python.org/issue44434

    A side effect of this change is the removal of the
    "PyThread_exit_thread called" threading debug log when using
    PYTHONTHREADDEBUG=1 environment variable.

    I never used PYTHONTHREADDEBUG. I just tried it and it produces tons
    of output in stdout about locks. It looks basically useless because it
    produces way too many logs, and it pollutes stdout (ex: most Python
    tests fail when it's enabled).

    This debug mode requires to build Python in debug mode (./configure
    --with-pydebug):

    https://docs.python.org/dev/using/configure.html#python-debug-build

    IMO there are enough external debugging tools to debug threading
    issues. Python no longer has to come with its built-in logs.

    I propose to deprecate the feature in Python 3.11 and remove it in 2
    releases (Python 3.13).

    Victor

    @vstinner vstinner added 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Jul 8, 2021
    @vstinner
    Copy link
    Member Author

    vstinner commented Jul 8, 2021

    I never ever used PYTHONTHREADDEBUG=1. I just tried:
    ---

    $ PYTHONTHREADDEBUG=1 ./python -m test -v test_sys
    PyThread_init_thread called
    PyThread_allocate_lock() -> 0x1f212c0
    PyThread_allocate_lock called
    PyThread_allocate_lock() -> 0x1f21300
    PyThread_allocate_lock called
    PyThread_allocate_lock() -> 0x1f21340
    PyThread_allocate_lock called
    PyThread_allocate_lock() -> 0x1f3ea70
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
    PyThread_release_lock(0x1f212c0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
    PyThread_release_lock(0x1f212c0) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
    PyThread_release_lock(0x1f21340) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
    PyThread_release_lock(0x1f21340) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
    PyThread_release_lock(0x1f21340) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
    PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
    PyThread_release_lock(0x1f21340) called
    (...)

    ======================================================================
    FAIL: test_changing_sys_stderr_and_removing_reference (test.test_sys.SizeofTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/vstinner/python/main/Lib/test/test_sys.py", line 1525, in test_changing_sys_stderr_and_removing_reference
        self.assertEqual(out, b"")
         ^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: b'PyThread_init_thread called\nPyThread_all[59813 chars]ed\n' != b''
    (...)
    PyThread_release_lock(0x2113660) called
    PyThread_free_lock(0x2113660) called
    PyThread_free_lock(0x7fc6e8001300) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
    PyThread_release_lock(0x1f212c0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
    PyThread_release_lock(0x1f212c0) called
    PyThread_free_lock(0x1f3ea70) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
    PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
    PyThread_release_lock(0x1f212c0) called
    PyThread_free_lock(0x1f212c0) called
    PyThread_free_lock(0x1f21300) called
    PyThread_free_lock(0x1f21340) called

    This debug mode produces so many logs that it looks basically useless :-(

    See also discussion at: #24241

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 6, 2021

    New changeset 4d77691 by Victor Stinner in branch 'main':
    bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
    4d77691

    @vstinner vstinner added 3.10 only security fixes labels Aug 6, 2021
    @miss-islington
    Copy link
    Contributor

    New changeset a11158e by Miss Islington (bot) in branch '3.10':
    bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
    a11158e

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 6, 2021

    PYTHONTHREADDEBUG is now deprecated in Python 3.10, and planned for removal in 3.12.

    If someone finds it useful, we can reconsider to keep the feature, but maybe redirect the output to stderr rather than stdout.

    @vstinner vstinner closed this as completed Aug 6, 2021
    @vstinner vstinner closed this as completed Aug 6, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    gpshead added a commit to gpshead/cpython that referenced this issue May 8, 2022
    @gpshead gpshead linked a pull request May 8, 2022 that will close this issue
    gpshead added a commit that referenced this issue May 9, 2022
    Remove the `PYTHONTHREADDEBUG` env var support.
    Remove no-op dprintf() macro calls.
    vstinner added a commit that referenced this issue May 10, 2022
    …LL (#92586)
    
    On Windows, PyThread_acquire_lock(), PyThread_acquire_lock_timed()
    and PyThread_release_lock() no longer check at runtime if the lock is
    not NULL.
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes 3.11 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    Successfully merging a pull request may close this issue.

    2 participants