This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Bind logger and waninigs modules for asyncio __del__ methods
Type: enhancement Stage: resolved
Components: asyncio Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: asvetlov Nosy List: asvetlov, gvanrossum, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2015-09-11 19:16 by asvetlov, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
asyncio_bind_modules.patch asvetlov, 2015-09-11 19:16 review
Messages (4)
msg250490 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2015-09-11 19:16
See https://github.com/KeepSafe/aiohttp/issues/497 for the reason.

Desctructors and descendant code (`loop.call_exception_handler()`) can be called on interpreter shutdown stage, which leads to printouts like 'None object has no attribute "errror"'.
msg250701 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-09-14 20:42
I don't like such "hacks". IMHO It's too late to try to log errors, to execute code to cleanup objects, etc. 

You should try to implement something in aiohttp or even in the application to cleanup objects at exit. For example, it's probably wrong if you still have tasks when the event loop is closed. Especially if tasks are still pending. See this part of the doc which lists "pending tasks at exit":
https://docs.python.org/dev/library/asyncio-dev.html#chain-coroutines-correctly

For "exceptions never consumed", I proposed a different enhancement in asyncio directly:
https://bugs.python.org/issue24598

What do you think?
msg250778 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2015-09-15 15:53
> You should try to implement something in aiohttp or even in the application to cleanup objects at exit. For example, it's probably wrong if you still have tasks when the event loop is closed. Especially if tasks are still pending.

The problem is for client API. For server I have more control and  implemented checks like you suggest.

For for client lazy user writes floppy code like:

    client = aiohttp.ClientSession()
    resp = yield from client.get(url)
    text = yield from resp.text()

Client is closed by GC, at the moment is not determined is loop has been closed or not (GC releases object in non-determenistic way).

So without changes I just cannot use `loop.call_exception_handler()` in `__del__` methods.

It's not only aiohttp issue but we should get rid of `call_exception_handler()` reports in asyncio itself for the same reason. 

We have `logger` and `warnings` modules usage e.g. in asyncio transports, it's not safe if transport is not closed properly before interpreter shutdown.

`subprecess` module has tricks for proper destruction like I've used in my patch.
msg254624 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2015-11-13 21:49
Yes, the approach use hackery shadows for binding global variables on shutdown stage.
The same I've found in subprocess module: see https://github.com/python/cpython/blob/master/Lib/subprocess.py#L1544

The problem is: I want to inform users about unclosed resources.
Printouts like
`TypeError: 'NoneType' object is not callable`
or 
`AttributeError: 'NoneType' object has no attribute 'warn'` are too confusing for casual developer.

On shutdown logger and warnings modules may be destroyed at the moment of __del__ call, so `warnings.warn`, `loop.call_exception_handler` etc. are forbidden.

As an option we can add atexit handler to store shutdown flag into loop object (I don't know other way to get info is interpreter in shutdown or not).

According to the flag we can eihter print some short text about non-closed resources or raise proper ResourceWarning along with comprehensive logging with stacktrace if PYTHONASYNCIO is on.

Maybe I've missed something and you can suggest better solution.
History
Date User Action Args
2022-04-11 14:58:20adminsetgithub: 69261
2017-12-20 21:11:54asvetlovsetstatus: open -> closed
resolution: wont fix
stage: patch review -> resolved
2015-11-13 21:49:43asvetlovsetmessages: + msg254624
2015-09-15 15:53:50asvetlovsetmessages: + msg250778
2015-09-14 20:42:02vstinnersetmessages: + msg250701
2015-09-11 19:16:05asvetlovcreate