classification
Title: AbstractEventLoop.shutdown_default_executor breaks backwards compatibility
Type: Stage: resolved
Components: asyncio Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: aeros, asvetlov, petr.viktorin, yselivanov
Priority: normal Keywords:

Created on 2020-05-28 09:31 by petr.viktorin, last changed 2020-06-09 18:06 by yselivanov. This issue is now closed.

Messages (3)
msg370182 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2020-05-28 09:31
In bpo-34037, AbstractEventLoop gained a new abstract method, shutdown_default_executor:
https://docs.python.org/dev/library/asyncio-eventloop.html#asyncio.loop.shutdown_default_executor

All AbstractEventLoop subclasses need to define this method, otherwise they are not compatible with Python 3.9's asyncio. It seems that the anyio and uvloop projects are affected: https://github.com/agronholm/anyio/issues/110

This is mentioned in What's New: https://docs.python.org/dev/whatsnew/3.9.html#changes-in-the-python-api

I'd like to make extra sure asyncio experts know about this backwards incompatibility.
Since asyncio is no longer provisional, should it break backwards compatibility with just a What's New entry?
msg370736 - (view) Author: Kyle Stanley (aeros) * (Python committer) Date: 2020-06-04 23:36
The main issue is that shutdown_default_executor() (or something that does the same thing) is necessary to ensure the resources of the event loop's ThreadPoolExecutor are finalized in a safe manner. Otherwise, it frequently results in dangling threads when run_in_executor() is used with the default executor, and was regularly doing so in the buildbots. So, I think this is a case where we ultimately have to break backwards compatibility in an upcoming version to fix a bug.

Also, the implementation of shutdown_default_executor() is fairly straightforward, and as far as I'm aware it's not incompatible with any of the listed projects that would need to implement it for 3.9. While inconvenient, I don't think the burden to address the compatibility break is at all high in this case.

> Since asyncio is no longer provisional, should it break backwards compatibility with just a What's New entry?

Do you have an alternative suggestion? Personally, I don't think a deprecation warning (or other) would make much sense here since it would be effectively leaving a resource leak bug as long as it isn't implemented.
msg371134 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2020-06-09 18:06
> Since asyncio is no longer provisional, should it break backwards compatibility with just a What's New entry?

Adding new APIs to asyncio can't be classified as a backward compatibility issue. Otherwise the development of it would stall and nobody would benefit from it. uvloop and anyio will be eventually updated to support new APIs, I don't see any problem here.
History
Date User Action Args
2020-06-09 18:06:27yselivanovsetstatus: open -> closed
resolution: not a bug
messages: + msg371134

stage: resolved
2020-06-04 23:36:04aerossetnosy: + aeros
messages: + msg370736
2020-05-28 09:31:51petr.viktorincreate