This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Add the ability to give custom names to threads created by ThreadPoolExecutor
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.11
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: RahulARanger, mark.dickinson
Priority: normal Keywords: patch

Created on 2021-11-02 09:15 by RahulARanger, last changed 2022-04-11 14:59 by admin. This issue is now closed.

File name Uploaded Description Edit RahulARanger, 2021-11-02 09:15 Test Case
Pull Requests
URL Status Linked Edit
PR 29359 closed RahulARanger, 2021-11-02 09:41
Repositories containing patches
Messages (8)
msg405496 - (view) Author: Tangellapalli Sai Hanuma Rahul (RahulARanger) * Date: 2021-11-02 09:15
Feature Request:

Where we can use custom Names for separate threads submitted in ThreadPoolExecutor. 

It achieves by sending the name string to _work_queue of ThreadPoolExecutor and then in _worker function modifies the name of the current thread (through name property).
msg405587 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2021-11-03 08:21
Feature requests are only considered for Python versions that have not yet been released, so only 3.11 should be listed in the "versions" for this ticket.
msg405931 - (view) Author: Tangellapalli Sai Hanuma Rahul (RahulARanger) * Date: 2021-11-08 06:51
ThreadPool handles tasks concurrently through Threads so users need not worry about the creation/deletion of Threads, it uses reuses threads whenever possible and it can handle any tasks. 

However, it should be possible for users to at least name/rename according to the tasks that the user submits into the ThreadPool.

* Good for Debugging (so user can know which thread has caused the issue)
* threadName attribute in logging's LogRecord can now use custom Name provided to the Task.

Until now thread_name_prefix parameter allows us to name the threads but those are not particular to the Task.


previously one could write .submit(function_name, *args, **kwargs)
but now one should write 
.submit(function_name, name_of_thread, *args, **kwargs)
name_of_thread can be None
msg405938 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-11-08 09:26
> previously one could write .submit(function_name, *args, **kwargs)
> but now one should write 
> .submit(function_name, name_of_thread, *args, **kwargs)
> name_of_thread can be None

This approach can't work, I'm afraid: it would be a backwards-incompatible change to the `submit` method signature, and would break many existing uses of submit.
msg406000 - (view) Author: Tangellapalli Sai Hanuma Rahul (RahulARanger) * Date: 2021-11-09 07:06
There is a new function submit_with_name in _base.Executor that can accept name parameters before args and kwargs, submit can continue to be used as before.

submit internally calls submit_with_name with name as None. Calling submit_with_name in Executor causes a NotImplementedError, it was only implemented in ThreadPoolExecutor, but it could be implemented in any Executor
msg406005 - (view) Author: Tangellapalli Sai Hanuma Rahul (RahulARanger) * Date: 2021-11-09 08:23
In summary;

Current Problem:
ThreadPoolExecutor handles all threads-related activities so the user needs to just submit tasks and set max_workers and other Executor settings if necessary. If ThreadPoolExecutor allowed us to name/rename threads, that would be great.

Of Course, there's a workaround that requires the user to change the name of the current Thread inside the Task, but that requires the user to access the current thread which I believe is not the goal of Thread Pool Executor.


Changes Made:
In this Pull request, _work_queue puts tuple of Future Object and name (name can be None).  i.e, typing.Tuple[_base.Future, typing. Optional[str]] which was just a _base.Future before.

So, inside the _worker function,
When it gets the elements and If its name is None, it will either use the thread_name_prefix or custom Name of task that was used before. else it sets the name of the current Thread with the given name.

Problems it solves

Currently, ThreadExecutor uses thread_name_prefix + index number for the Internal Thread which is not enough for debugging when we try to submit different types of Tasks (types here mean general callable). 

So, using this Optional Feature which allows users to submit tasks with names it allows them to easily find issues using the name of Thread either in Logger or in the debugger.
msg406036 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-11-09 17:51
Sorry Rahul: I'm not the right person to help you push this forward.

I'm sympathetic to the problem: I've encountered similar issues in "Real Code", where we needed to associate log outputs generated by worker pool threads with the actual tasks that generated those logs. But I'm not convinced either that setting the thread name is the right mechanism to get that association (it doesn't extend nicely to other executor types, for example), or that the API you propose is the right one (I find the duplication between `submit` and `submit_with_name` to be a bit much).

I'm wondering whether there's some way that executors could use contextvars to provide a per-task context. Then a task "name" could potentially be part of that context, and possibly you could write a custom log handler that read the name from the context when emitting log messages.

If you want to find someone to help push this forward, it may be worth posting on the python-ideas mailing list to get discussion going.
msg406073 - (view) Author: Tangellapalli Sai Hanuma Rahul (RahulARanger) * Date: 2021-11-10 05:51
Yes, it's true, it's so naive method, Contextvars! may be inside the Future Object?
Date User Action Args
2022-04-11 14:59:51adminsetgithub: 89852
2021-11-10 05:51:06RahulARangersetstatus: open -> closed
resolution: rejected
messages: + msg406073

stage: patch review -> resolved
2021-11-09 17:51:14mark.dickinsonsetmessages: + msg406036
2021-11-09 08:23:42RahulARangersetmessages: + msg406005
2021-11-09 07:06:02RahulARangersetmessages: + msg406000
2021-11-08 09:26:24mark.dickinsonsetnosy: + mark.dickinson
messages: + msg405938
2021-11-08 08:53:34AlexWaygoodsettitle: Custom Name for ThreadPoolExecutor -> Add the ability to give custom names to threads created by ThreadPoolExecutor
2021-11-08 06:51:21RahulARangersetmessages: + msg405931
2021-11-03 08:21:43AlexWaygoodsetnosy: - AlexWaygood
2021-11-03 08:21:30AlexWaygoodsetnosy: + AlexWaygood

messages: + msg405587
versions: + Python 3.11, - Python 3.6, Python 3.7, Python 3.8, Python 3.9
2021-11-03 06:46:25RahulARangersetversions: - Python 3.10, Python 3.11
2021-11-02 09:41:37RahulARangersethgrepos: - hgrepo410
2021-11-02 09:41:23RahulARangersetkeywords: + patch
hgrepos: + hgrepo410
stage: patch review
pull_requests: + pull_request27619
2021-11-02 09:15:22RahulARangercreate