classification
Title: threading.Thread: use target name if the name parameter is omitted
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ammar2, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2020-09-22 11:47 by vstinner, last changed 2020-10-12 08:58 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22357 merged vstinner, 2020-09-22 11:55
Messages (11)
msg377314 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-22 11:47
When debugging race conditions in a multithreaded application with many threads, it's hard to debug when threads have generic names like "Thread-3".

I propose to use the target name in threading.Thread constructor if the name parameter is omitted.


See for example bpo-41739 with "Dangling thread: <Thread(Thread-3, started daemon 4396088817936)>": the "Thread-3" name is not helpful. I suspect that the bug comes from threading.Thread(target=remove_loop, args=(fn, del_count)): with my proposed change, the thread would be called "target=remove_loop", which is more helpful than "Thread".

Fall back on _newname() as usual if target.__name__ attribute does not exist.

Attached PR implements this idea.
msg377320 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-22 13:31
And what if run different threads with the same target or with different targets with the same name? Would not "Thread-3" be more useful in this case?

for i in range(10):
    Thread(target=print, args=(i,)).start()

for i in range(10):
    def doit(i=i):
        print(i)
    Thread(target=doit).start()
msg377328 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-22 14:32
> And what if run different threads with the same target or with different targets with the same name?

If multiple Thread objects using the same target (or different targets with the same name), they all get the same name using my PR 22357.

> Would not "Thread-3" be more useful in this case?

Well, that's an open question. In my experience, "Thread" name is pretty useless.

What if I modify my PR to add "-{counter}" (ex: "func-3" for target.__name__="func") to the default name? Reuse _counter().

Pseudo-code:

        self._name = str(name)
        try:
            base_name = target.__name__
        except AttributeError:
            base_name = "Thread"
        if not self._name:
            self._name = "{base_name)-{_counter()}"
msg377330 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-22 14:40
See also bpo-15500 "Python should support exporting thread names to the OS".
msg377333 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2020-09-22 14:54
Having the target in the thread name definitely seems like a good idea and would help ease debugging, not just for our tests but when printing thread objects in general. 

I would agree with the suggestion of placing the counter in there for when multiple threads get spawned with the same target or maybe even using names like:

  Thread-1 (doit)
  Thread-2
  Thread-3 (print)

might be better just for people who are familiar with them.
msg377405 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-23 16:25
I rewrote my PR to generate names like "Thread-3 (func)", rather than just "func". So if two target functions have the same name, or if two threads use the same target, it remains possible to distinguish the two threads.
msg377407 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-09-23 16:43
Ideally, I would prefer separate counters for different names, and omitting a number for the first thread with unique name:

  doit
  doit-2
  Thread-1
  Thread-2
  print
  print-2
  doit-3

But it is not feasible. It would require supporting a cache which can grow with every new thread and non-trivial code to avoid race conditions. So I am fine with Ammar's idea.
msg377411 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-23 16:57
> Ideally, I would prefer separate counters for different names

IMO if you want to go at the level of details, I suggest you to generate yourself thread names:

threads = [threading.Thread(name=f"MyThread-{i}") for i in range(1, 6)]

Maintaining a list of thread names sounds overkill to me. It would be quite complicated and would increase the memory footprint, for little benefit.
msg377421 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-23 21:21
New changeset 98c16c991d6e70a48f4280a7cd464d807bdd9f2b by Victor Stinner in branch 'master':
bpo-41833: threading.Thread now uses the target name (GH-22357)
https://github.com/python/cpython/commit/98c16c991d6e70a48f4280a7cd464d807bdd9f2b
msg377422 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-09-23 21:23
I merged my change with "Thread-1 (func)" format: add the target name, but keep "Thread-3" to distinguish two different threads with the same target name.
msg378479 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-10-12 08:58
Cool, my threading change works as expected! The name of the target function was logged in bpo-41739 issue.

https://buildbot.python.org/all/#/builders/509/builds/172

0:00:28 load avg: 3.58 [ 36/424/1] test_logging failed (env changed)
Warning -- threading_cleanup() failed to cleanup 2 threads (count: 2, dangling: 3)
Warning -- Dangling thread: <Thread(Thread-4 (removeTarget), started daemon 4396862667024)>
Warning -- Dangling thread: <Thread(Thread-11 (removeTarget), started daemon 4396595259664)>
Warning -- Dangling thread: <_MainThread(MainThread, started 4396920310576)>

See "<Thread(Thread-4 (removeTarget), started daemon 4396862667024)>": the two leaked threads are running the removeTarget() function which come from MemoryHandlerTest.test_race_between_set_target_and_flush().

By the way, Serhiy was right about the name of having an unique repr() even if the target is the same, since it's the case here ;-) Well "4396862667024" is different than "4396595259664", but it's more convinient to have short numbers like "Thread-4" and "Thread-11".
History
Date User Action Args
2020-10-12 08:58:07vstinnersetmessages: + msg378479
2020-09-23 21:23:21vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg377422

stage: patch review -> resolved
2020-09-23 21:21:29vstinnersetmessages: + msg377421
2020-09-23 16:57:45vstinnersetmessages: + msg377411
2020-09-23 16:43:00serhiy.storchakasetmessages: + msg377407
2020-09-23 16:25:34vstinnersetmessages: + msg377405
2020-09-22 14:54:44ammar2setnosy: + ammar2
messages: + msg377333
2020-09-22 14:40:49vstinnersetmessages: + msg377330
2020-09-22 14:32:03vstinnersetmessages: + msg377328
2020-09-22 13:31:41serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg377320
2020-09-22 11:55:41vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request21395
2020-09-22 11:47:35vstinnercreate