Author yselivanov
Recipients aeros, asvetlov, primal, yselivanov
Date 2019-11-02.20:54:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1572728069.97.0.103098961811.issue32309@roundup.psfhosted.org>
In-reply-to
Content
>  From my understanding, the executor classes are designed around spawning the threads (or processes in the case of ProcessPoolExecutor) as needed up to max_workers, rather than spawning them upon startup. The asynchronous spawning of threads or processes would also not be compatible with the executor subclasses as far as I can tell. 

> I can start working on a draft/prototype for a design. It will likely take more time to implement this, but it will give us the chance to have a native asyncio ThreadPool that doesn't directly rely upon the API in concurrent.futures. 

No, that would be too much work. Writing a thread pool or process pool from scratch is an extremely tedious task and it will take us years to stabilize the code. It's not simple.

We should design *our* API correctly though. And that means that we can't initialize our pools in __init__.

Something along these lines would work:

    class ThreadPool:
      async def start(): ...
      async def __aenter__(self): 
        await self.start()
        return self
      
      async def aclose(): ...
      async def __aexit__(self, *exc):
        await self.aclose()


> Let me know if you approve of this idea Yury and Andrew. It's quite a bit more involved than implementing a simple high level version of loop.run_in_executor(), but I think it would prove to be worthwhile in the long term.

It shouldn't be much harder than run_in_executor() as we continue to rely on concurrent.future (at least for the first version).

We need to start the discussion about this API on discourse.  Please give me a few days to organize that.
History
Date User Action Args
2019-11-02 20:54:30yselivanovsetrecipients: + yselivanov, asvetlov, primal, aeros
2019-11-02 20:54:29yselivanovsetmessageid: <1572728069.97.0.103098961811.issue32309@roundup.psfhosted.org>
2019-11-02 20:54:29yselivanovlinkissue32309 messages
2019-11-02 20:54:29yselivanovcreate