classification
Title: asyncio: emit ResourceWarning warnings if transports/event loops are not explicitly closed
Type: Stage:
Components: asyncio Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, python-dev, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2015-01-14 23:18 by vstinner, last changed 2015-01-29 16:55 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
resource_warnings.patch vstinner, 2015-01-14 23:18 review
server_close.patch vstinner, 2015-01-16 00:19
Messages (7)
msg234043 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-14 23:18
I propose to add destructors to transports and event loops which emit a ResourceWarning if they are not closed.

The change should help to detect resource leaks and bugs.

Attached patch implements this issue. It only adds destructors on Python 3.4 and later, because older Python versions don't implement the PEP 442 (Safe object finalization) and so objects part of reference cycle would never be deleted.

The patch adds a new _closed attribute to BaseSubprocessTransport and _SSLProtocolTransport classes, to track if the transport was closed or not.

The patch should help to find bugs like this one:
https://code.google.com/p/tulip/issues/detail?id=218
msg234071 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-01-15 12:25
New changeset 2f13d53f4680 by Victor Stinner in branch '3.4':
Issue #23243: Fix asyncio._UnixWritePipeTransport.close()
https://hg.python.org/cpython/rev/2f13d53f4680

New changeset aef0f9b4e729 by Victor Stinner in branch '3.4':
Issue #23243: Close explicitly event loops in asyncio tests
https://hg.python.org/cpython/rev/aef0f9b4e729

New changeset f9b127188d43 by Victor Stinner in branch '3.4':
Issue #23243: Close explicitly transports in asyncio tests
https://hg.python.org/cpython/rev/f9b127188d43
msg234074 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-15 13:35
I fixed a lot of methods and tests to ensure that all event loops and transports are properly closed. These changes were required to prepare the integration of this change.

Many changed were real bug fixes: closing explicitly tranports on error avoids to leak resources.

Some ResourceWarning are still emited on SSL server tests.
msg234103 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-16 00:19
> Some ResourceWarning are still emited on SSL server tests.

Attached server_close.patch is a work-in-process patch to fix this issue on Linux.

If I understood correctly, there are two issues:

- SelectorEventLoop._accept_connection() doesn't care of the creation of the transport failed
- SSLProtocol.eof_received() does not notify the waiter that EOF was received

So an application cannot control SSL incoming connections which fail during the handshake? Control: log errors, do something with the socket, etc.

See also the STARTTLS feature request:
https://code.google.com/p/tulip/issues/detail?id=79
msg234858 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-27 22:41
> - SelectorEventLoop._accept_connection() doesn't care of the creation of the transport failed

Fixing this issue requires to change the design of asyncio, so I chose to open a new issue: issue #23333.
msg234979 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-01-29 16:54
New changeset 543f770f62f0 by Victor Stinner in branch '3.4':
Issue #23243, asyncio: Emit a ResourceWarning when an event loop or a transport
https://hg.python.org/cpython/rev/543f770f62f0
msg234980 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-29 16:55
Ok, I commited my change adding destructors on Python 3.4 and newer to emit warnings when event loops or transports are not explicitly closed.
History
Date User Action Args
2015-01-29 16:55:05vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg234980
2015-01-29 16:54:02python-devsetmessages: + msg234979
2015-01-27 22:41:30vstinnersetmessages: + msg234858
2015-01-16 00:19:31vstinnersetfiles: + server_close.patch

messages: + msg234103
2015-01-15 13:35:23vstinnersetmessages: + msg234074
2015-01-15 12:25:44python-devsetnosy: + python-dev
messages: + msg234071
2015-01-14 23:18:04vstinnercreate