Author vstinner
Recipients Ben.Darnell, Big Stone, asvetlov, lukasz.langa, vstinner, yselivanov
Date 2019-06-24.10:50:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1561373453.51.0.252049655427.issue37373@roundup.psfhosted.org>
In-reply-to
Content
Ben:
> Concretely, this is a concern for Tornado (which requires add_reader()) and applications in the scientific python community (including Jupyter) which depend on it.

If you need add_reader/add_writer in Python 3.8, you can switch the default event loop to selector at the beginning of your application:

if sys.platform == 'win32':
  asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())


Yury:
> If there's no native API for that, I guess we can spawn a thread with a 'select()' call to emulate this API?

There is no native API because IOCP design is to run an asynchronous read/write and then wait for  its completion. Unix select() has the opposite design: check if a file descriptor is read for read/write.

Reimplementing add_reader/add_writer using a single select() call in a thread sounds like a good idea, but we have to make sure that it can be stopped whenever using a dedicated "self-pipe" (to awake the blocked select(), so loop.close() can stop this thread.

See attached proof-of-concept: selector_thread.py: run a selector in a separated thread which pass pack events to the loop using call_soon().

I would prefer to use a single selector to better scale with the number of FD.

Note: On Windows, select() only supports sockets.

Note: select.select() may be extended to support more than 512 sockets on Windows, see bpo-28708 :-)


Yury:
> Another question: if we fix this, would you allow this to go in beta2/3?  Strictly speaking it's going to be a new functionality.

It's a new feature, so it can wait for Python 3.9 :-)

I don't see any regression here, as soon as you can opt-in for SelectorEventLoop.
History
Date User Action Args
2019-06-24 10:50:53vstinnersetrecipients: + vstinner, asvetlov, lukasz.langa, Ben.Darnell, yselivanov, Big Stone
2019-06-24 10:50:53vstinnersetmessageid: <1561373453.51.0.252049655427.issue37373@roundup.psfhosted.org>
2019-06-24 10:50:53vstinnerlinkissue37373 messages
2019-06-24 10:50:53vstinnercreate