This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add name to process and thread pool
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Raz Manor, matrixise, pitrou, xtreak
Priority: normal Keywords: patch

Created on 2018-10-16 09:47 by Raz Manor, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 9906 closed Raz Manor, 2018-10-16 09:57
Messages (6)
msg327819 - (view) Author: Raz Manor (Raz Manor) Date: 2018-10-16 10:40
Add a human friendly names to the threads opened by multiprocessing.pool.Pool and multiprocessing.pool.ThreadPool objects.

Sample usage:
ThreadPool(name="ClientsPool", processes=8)
msg327865 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-17 06:29
Thanks for the report. I think using current_thread().name gives thread name with number that is useful. I don't know the exact use case or maybe I am misunderstanding the use case and perhaps adding a script where your PR applies with the output will be helpful. I will resort to feedback from others. Adding Antoine as per the expert index for multiprocessing module. Antoine, feel free to remove yourself if this is irrelevant.


# bpo34720.py

from multiprocessing.pool import ThreadPool

def f(x):
    from threading import current_thread
    print(current_thread().name)
    return x*x

if __name__ == '__main__':
    with ThreadPool(5) as p:
        print(p.map(f, [1, 2, 3]))


$ ./python.exe ../backups/bpo34720.py
Thread-1
Thread-1
Thread-2
[1, 4, 9]


# With PR and name as "custom-name" for ThreadPool

from multiprocessing.pool import ThreadPool

def f(x):
    from threading import current_thread
    print(current_thread().name)
    return x*x

if __name__ == '__main__':
    with ThreadPool(5, name="custom-name") as p:
        print(p.map(f, [1, 2, 3]))

git:(pr_9906) ./python.exe ../backups/bpo34720.py
custom-name-Worker-0
custom-name-Worker-1
custom-name-Worker-2
[1, 4, 9]
msg327900 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-10-17 14:32
Hi Raz,

1. Please could you sign the CLA?
2. About the name, why did you use these names? Is there a discussion somewhere about the names?

Thank you
msg328196 - (view) Author: Raz Manor (Raz Manor) Date: 2018-10-21 07:48
The default name of the threads does not allow differentiation between pool threads and other threads. This problem is more notable when you have several thread pools. Also, since there are some management threads to the pool, and one might want to know which is which. It was very helpful for me while debugging some memory issues in my project, this is where the idea came from.

The names themselves were my idea, but I welcome any change to them.
msg328199 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2018-10-21 08:36
Thanks for posting this.  I think this is a good idea, will take a look at the PR later.
msg328239 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-10-22 07:26
Thanks for clarifying the use case. I think it's a good idea with multiple thread pools and below outputs for the program with the PR is more easier to debug than the current code. 

# ../backups/bpo34720.py

from multiprocessing.pool import ThreadPool

def f(x):
    from threading import current_thread
    print(current_thread().name)
    return x*x

if __name__ == '__main__':
    p1 = ThreadPool(5, name="process-1")
    p1.map_async(f, [1, 2, 3])
    p1.close()

    p2 = ThreadPool(5, name="process-2")
    p2.map_async(f, [1, 2, 3])
    p2.close()

    p1.join()
    p2.join()

# With patch

$ ./python.exe ../backups/bpo34720.py
process-1-Worker-0
process-1-Worker-1
process-1-Worker-0
process-2-Worker-0
process-2-Worker-1
process-2-Worker-0

# Without patch and removing name parameter

$ python3.7 ../backups/bpo34720.py
Thread-1
Thread-1
Thread-1
Thread-9
Thread-9
Thread-9
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79177
2018-10-22 07:26:52xtreaksetmessages: + msg328239
2018-10-21 08:36:03pitrousetmessages: + msg328199
2018-10-21 07:48:16Raz Manorsetmessages: + msg328196
2018-10-18 12:31:23pitrousetversions: + Python 3.8, - Python 2.7, Python 3.7
2018-10-17 14:32:56matrixisesetnosy: + matrixise
messages: + msg327900
2018-10-17 06:29:21xtreaksetnosy: + xtreak, pitrou
messages: + msg327865
2018-10-16 10:40:43Raz Manorsetmessages: + msg327819
2018-10-16 09:57:12Raz Manorsetkeywords: + patch
stage: patch review
pull_requests: + pull_request9268
2018-10-16 09:47:56Raz Manorcreate