Message403007
Currently the only way to use a class other than `threading.Thread` with `concurrent.futures.ThreadPoolExecutor` is to extend `ThreadPoolExecutor` and override the private method `_adjust_thread_count()`.
For example, suppose I have a class that applies some custom logic when running code in a new thread:
```
class MyThread(threading.Thread):
def run(self):
with some_important_context():
super().run()
```
Using this class with `ThreadPoolExecutor` requires me to write the following code:
```
class MyThreadPoolExecutor(ThreadPoolExecutor):
def _adjust_thread_count(self):
# if idle threads are available, don't spin new threads
if self._idle_semaphore.acquire(timeout=0):
return
# When the executor gets lost, the weakref callback will wake up
# the worker threads.
def weakref_cb(_, q=self._work_queue):
q.put(None)
num_threads = len(self._threads)
if num_threads < self._max_workers:
thread_name = '%s_%d' % (self._thread_name_prefix or self,
num_threads)
t = MyThread(name=thread_name, target=_worker,
args=(weakref.ref(self, weakref_cb),
self._work_queue,
self._initializer,
self._initargs))
t.start()
self._threads.add(t)
_threads_queues[t] = self._work_queue
with MyThreadPoolExecutor(max_workers=1) as executor:
future = executor.submit(pow, 323, 1235)
print(future.result())
```
That's a bummer, because now I have to maintain this method if there are upstream fixes/changes. I also can't count on it existing in the future, since it's a private method.
In other words, `ThreadPoolExecutor` is not composable, and extending it to use a custom `Thread` class is neither safe nor maintainable.
Here's what I'd like to be able to do instead:
```
with ThreadPoolExecutor(max_workers=1, thread_class=MyThread) as executor:
future = executor.submit(pow, 323, 1235)
print(future.result())
``` |
|
Date |
User |
Action |
Args |
2021-10-01 10:37:53 | erickpeirson | set | recipients:
+ erickpeirson |
2021-10-01 10:37:53 | erickpeirson | set | messageid: <1633084673.43.0.269469120691.issue45339@roundup.psfhosted.org> |
2021-10-01 10:37:53 | erickpeirson | link | issue45339 messages |
2021-10-01 10:37:53 | erickpeirson | create | |
|