classification
Title: Add support for io_uring to cpython
Type: enhancement Stage: resolved
Components: IO Versions: Python 3.10
process
Status: closed Resolution: later
Dependencies: Superseder:
Assigned To: Nosy List: ammar2, benjamin.peterson, cooperlees, corona10, giampaolo.rodola, ronaldoussoren, terry.reedy
Priority: normal Keywords:

Created on 2020-07-10 19:54 by cooperlees, last changed 2020-07-18 07:55 by ronaldoussoren. This issue is now closed.

Messages (7)
msg373477 - (view) Author: Cooper Lees (cooperlees) * Date: 2020-07-10 19:54
Would adding support for io_uring in Linux to stadlib IO and/or asyncio make sense?

More info on io_uring:
- https://kernel.dk/io_uring.pdf
- https://lwn.net/Articles/810414/
msg373699 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-07-15 15:20
How stable is the io_uring API?

The stdlib evolves slowly, I'm not sure if the io_uring interface has matured enough to make it acceptable that updates to the stdlib bindings for that API are only done in new major releases of CPython (about every 18 months). 

It is probably better to create a PyPI package for this API. This package can then also provide an asyncio eventloop implementation.
msg373703 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2020-07-15 17:08
I agree that adding stdlib support is premature. io_uring has a large and growing API surface. It will take some experimentation to see what the best way to expose its power to Python is.
msg373704 - (view) Author: Cooper Lees (cooperlees) * Date: 2020-07-15 17:35
Totally agree with a separate module first to POC. I should have stated that. Main goal was to open this issue to start discussions :)
msg373734 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2020-07-16 01:13
See also: https://github.com/python-trio/trio/issues/932 which contains a link to a library with cffi bindings to io_uring
msg373868 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-07-18 01:43
I think that the answer is maybe, eventually, and if so, an ioring module that any event framework can use and a separate asyncio module for its use with asyncio.  I say maybe because the lwn article suggests that additions and revisions might continue for a decade or more.  A pypi py_iorin wrapper for lib_ioring might need revision with each Linux x.y release for a long time.

The opening question could have been raised on the python-ideas or possibly pydev lists.  Since this list is for patches to cpython, creating and maintaining a py_ioring package on pypi should be discussed elsewhere.  In the meanwhile, this issue could be closed as 'later' ("Issue is to be worked on in a later release cycle.").
msg373884 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-07-18 07:55
I'm closing this as "later" as the consensus is that this might be a useful binding to have, but io_uring is evolving too fast at the moment. 

It is better to develop these bindings on PyPI, which also makes it easier to iterate on the design.
History
Date User Action Args
2020-07-18 07:55:00ronaldoussorensetstatus: open -> closed
resolution: later
messages: + msg373884

stage: resolved
2020-07-18 01:43:48terry.reedysetnosy: + terry.reedy
messages: + msg373868
2020-07-18 01:03:30giampaolo.rodolasetnosy: + giampaolo.rodola
2020-07-16 02:39:06corona10setnosy: + corona10
2020-07-16 01:13:22ammar2setnosy: + ammar2
messages: + msg373734
2020-07-15 17:35:49cooperleessetmessages: + msg373704
2020-07-15 17:08:02benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg373703
2020-07-15 15:20:55ronaldoussorensetnosy: + ronaldoussoren
messages: + msg373699
2020-07-10 19:54:06cooperleescreate