Message341917
When I first wrote and started using ThreadPoolExecutor, I had a lot of code like this:
with ThreadPoolExecutor(max_workers=500) as e:
e.map(download, images)
I didn't expect that `images` would be a large list but, if it was, I wanted all of the downloads to happen in parallel.
I didn't want to have to explicitly take into account the list size when starting the executor (e.g. max_works=min(500, len(images))) but I also didn't want to create 500 threads up front when I only needed a few.
My use case involved transient ThreadPoolExecutors so I didn't have to worry about idle threads.
In principle, I'd be OK with trying to avoid unnecessary thread creation if the implementation can be simple and efficient enough.
https://github.com/python/cpython/pull/6375 seems simple enough but I haven't convinced myself that it works yet ;-) |
|
Date |
User |
Action |
Args |
2019-05-08 17:58:29 | bquinlan | set | recipients:
+ bquinlan, pitrou, methane, torsten, josh.r, Matt Spitz, tomMoral, dmacnet, iunknwn |
2019-05-08 17:58:29 | bquinlan | set | messageid: <1557338309.87.0.315116847878.issue24882@roundup.psfhosted.org> |
2019-05-08 17:58:29 | bquinlan | link | issue24882 messages |
2019-05-08 17:58:29 | bquinlan | create | |
|