Author tilsche
Recipients Matt Spitz, bquinlan, dmacnet, iunknwn, josh.r, methane, pitrou, tilsche, tomMoral, torsten
Date 2019-05-18.12:50:35
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1558183835.48.0.801212148637.issue24882@roundup.psfhosted.org>
In-reply-to
Content
We ran into this issue in the context of asyncio which uses an internal ThreadPoolExecutor to provide an asynchronous getaddrinfo / getnameinfo.

We observed an async application spawned more and more threads through several reconnects. With a maximum of 5 x CPUs these were dozens of threads which easily looked like a resource leak.

At least in this scenario I would strongly prefer to correctly reuse idle threads. 

Spawning all possible threads on initialization in such a transparent case would be quite bad. Imagine having a process-parallel daemon that running a apparently single-threaded asyncio loop but then getting these executors for doing a single asyncio.getaddrinfo. Now you run 80 instances on an 80 core machine you get 32.000 extra implicit threads.

Now you can argue whether the default executor in asyncio is good as is, but if the executors properly reuse threads, it would be quite unlikely to be a practical problem.
History
Date User Action Args
2019-05-18 12:50:35tilschesetrecipients: + tilsche, bquinlan, pitrou, methane, torsten, josh.r, Matt Spitz, tomMoral, dmacnet, iunknwn
2019-05-18 12:50:35tilschesetmessageid: <1558183835.48.0.801212148637.issue24882@roundup.psfhosted.org>
2019-05-18 12:50:35tilschelinkissue24882 messages
2019-05-18 12:50:35tilschecreate