Message355881
> From my understanding, the executor classes are designed around spawning the threads (or processes in the case of ProcessPoolExecutor) as needed up to max_workers, rather than spawning them upon startup. The asynchronous spawning of threads or processes would also not be compatible with the executor subclasses as far as I can tell.
> I can start working on a draft/prototype for a design. It will likely take more time to implement this, but it will give us the chance to have a native asyncio ThreadPool that doesn't directly rely upon the API in concurrent.futures.
No, that would be too much work. Writing a thread pool or process pool from scratch is an extremely tedious task and it will take us years to stabilize the code. It's not simple.
We should design *our* API correctly though. And that means that we can't initialize our pools in __init__.
Something along these lines would work:
class ThreadPool:
async def start(): ...
async def __aenter__(self):
await self.start()
return self
async def aclose(): ...
async def __aexit__(self, *exc):
await self.aclose()
> Let me know if you approve of this idea Yury and Andrew. It's quite a bit more involved than implementing a simple high level version of loop.run_in_executor(), but I think it would prove to be worthwhile in the long term.
It shouldn't be much harder than run_in_executor() as we continue to rely on concurrent.future (at least for the first version).
We need to start the discussion about this API on discourse. Please give me a few days to organize that. |
|
Date |
User |
Action |
Args |
2019-11-02 20:54:30 | yselivanov | set | recipients:
+ yselivanov, asvetlov, primal, aeros |
2019-11-02 20:54:29 | yselivanov | set | messageid: <1572728069.97.0.103098961811.issue32309@roundup.psfhosted.org> |
2019-11-02 20:54:29 | yselivanov | link | issue32309 messages |
2019-11-02 20:54:29 | yselivanov | create | |
|