This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author bquinlan
Recipients Matt Spitz, bquinlan, dmacnet, iunknwn, josh.r, methane, pitrou, tomMoral, torsten
Date 2019-05-08.17:58:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1557338309.87.0.315116847878.issue24882@roundup.psfhosted.org>
In-reply-to
Content
When I first wrote and started using ThreadPoolExecutor, I had a lot of code like this:

with ThreadPoolExecutor(max_workers=500) as e:
  e.map(download, images)

I didn't expect that `images` would be a large list but, if it was, I wanted all of the downloads to happen in parallel.

I didn't want to have to explicitly take into account the list size when starting the executor (e.g. max_works=min(500, len(images))) but I also didn't want to create 500 threads up front when I only needed a few.

My use case involved transient ThreadPoolExecutors so I didn't have to worry about idle threads.

In principle, I'd be OK with trying to avoid unnecessary thread creation if the implementation can be simple and efficient enough.

https://github.com/python/cpython/pull/6375 seems simple enough but I haven't convinced myself that it works yet ;-)
History
Date User Action Args
2019-05-08 17:58:29bquinlansetrecipients: + bquinlan, pitrou, methane, torsten, josh.r, Matt Spitz, tomMoral, dmacnet, iunknwn
2019-05-08 17:58:29bquinlansetmessageid: <1557338309.87.0.315116847878.issue24882@roundup.psfhosted.org>
2019-05-08 17:58:29bquinlanlinkissue24882 messages
2019-05-08 17:58:29bquinlancreate