classification
Title: Deprecate thread debugging PYTHONTHREADDEBUG=1
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, vstinner
Priority: normal Keywords: patch

Created on 2021-07-08 09:23 by vstinner, last changed 2021-08-06 12:14 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27065 merged vstinner, 2021-07-08 10:28
PR 27624 merged miss-islington, 2021-08-06 11:11
Messages (5)
msg397130 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-07-08 09:23
Copy of my email to python-dev:
https://mail.python.org/archives/list/python-dev@python.org/thread/NMLGCDRUKLZSTK4UICJTKR54WRXU2ZGJ/

Hi,

Does anyone use threading debug PYTHONTHREADDEBUG=1 env var on a
Python debug build? If not, can I just remove it?

--

To fix a race condition at thread exit on Linux using the glibc, I
removed calls to pthread_exit() (PyThread_exit_thread()) in the
_thread module:

   https://bugs.python.org/issue44434

A side effect of this change is the removal of the
"PyThread_exit_thread called" threading debug log when using
PYTHONTHREADDEBUG=1 environment variable.

I never used PYTHONTHREADDEBUG. I just tried it and it produces tons
of output in stdout about locks. It looks basically useless because it
produces way too many logs, and it pollutes stdout (ex: most Python
tests fail when it's enabled).

This debug mode requires to build Python in debug mode (./configure
--with-pydebug):

   https://docs.python.org/dev/using/configure.html#python-debug-build

IMO there are enough external debugging tools to debug threading
issues. Python no longer has to come with its built-in logs.

I propose to deprecate the feature in Python 3.11 and remove it in 2
releases (Python 3.13).

Victor
msg397131 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-07-08 09:24
I never ever used PYTHONTHREADDEBUG=1. I just tried:
---
$ PYTHONTHREADDEBUG=1 ./python -m test -v test_sys
PyThread_init_thread called
PyThread_allocate_lock() -> 0x1f212c0
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f21300
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f21340
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f3ea70
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
(...)
======================================================================
FAIL: test_changing_sys_stderr_and_removing_reference (test.test_sys.SizeofTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_sys.py", line 1525, in test_changing_sys_stderr_and_removing_reference
    self.assertEqual(out, b"")
     ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: b'PyThread_init_thread called\nPyThread_all[59813 chars]ed\n' != b''
(...)
PyThread_release_lock(0x2113660) called
PyThread_free_lock(0x2113660) called
PyThread_free_lock(0x7fc6e8001300) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_free_lock(0x1f3ea70) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_free_lock(0x1f212c0) called
PyThread_free_lock(0x1f21300) called
PyThread_free_lock(0x1f21340) called
---

This debug mode produces so many logs that it looks basically useless :-(

See also discussion at: https://github.com/python/cpython/pull/24241
msg399071 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-08-06 11:11
New changeset 4d77691172aae81bdcbb0ea75839d0e896c43781 by Victor Stinner in branch 'main':
bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
https://github.com/python/cpython/commit/4d77691172aae81bdcbb0ea75839d0e896c43781
msg399074 - (view) Author: miss-islington (miss-islington) Date: 2021-08-06 11:33
New changeset a11158ecef8cff795f7db8f4047cbd20cc9cf37e by Miss Islington (bot) in branch '3.10':
bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
https://github.com/python/cpython/commit/a11158ecef8cff795f7db8f4047cbd20cc9cf37e
msg399077 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-08-06 12:14
PYTHONTHREADDEBUG is now deprecated in Python 3.10, and planned for removal in 3.12.

If someone finds it useful, we can reconsider to keep the feature, but maybe redirect the output to stderr rather than stdout.
History
Date User Action Args
2021-08-06 12:14:30vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg399077

stage: patch review -> resolved
2021-08-06 11:33:15miss-islingtonsetmessages: + msg399074
2021-08-06 11:13:34vstinnersetversions: + Python 3.10
2021-08-06 11:11:20miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26118
2021-08-06 11:11:19vstinnersetmessages: + msg399071
2021-07-08 10:28:01vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request25618
2021-07-08 09:24:31vstinnersetmessages: + msg397131
2021-07-08 09:23:09vstinnercreate