classification
Title: asyncio create_connection resource warning
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, landersson, python-dev, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2014-04-01 03:04 by landersson, last changed 2014-06-03 22:21 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
timeout.py landersson, 2014-04-01 03:04 resource usage issue reproduction code
create_connection_close.patch vstinner, 2014-04-04 16:39 review
close2.patch vstinner, 2014-04-04 16:52 review
close-3.patch vstinner, 2014-04-09 01:04 review
Messages (8)
msg215291 - (view) Author: Lars Andersson (landersson) Date: 2014-04-01 03:04
The attached code generates an unclosed socket ResourceWarning when timing out trying to connect to an unreachable address.

Probably not terribly serious, but in my case, it generates distracting warnings during unit testing.

I have looked briefly at the asyncio code but it's not immediately obvious to me how to fix it.
msg215534 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-04-04 16:39
Here is a patch for Python 3.5. It should be applied to Python 3.4 too.
msg215535 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-04-04 16:52
BaseEventLoop.create_datagram_endpoint() and _UnixSelectorEventLoop.create_unix_server() have the same bug.

close2.patch fixes these methods but also modify socketpair() to ensure that the 2 sockets are closed on error.

I didn't audit the whole asyncio module.

Note: BaseEventLoop.create_server() uses a different approach: a "completed" flag with a try/finally block.
msg215674 - (view) Author: Lars Andersson (landersson) Date: 2014-04-07 00:52
Thanks Victor, that fixes my problem.

I've started using tulip/master as part of my project as that also solves other issues I have with the default asyncio of python 3.4.0, but hopefully this fix will into tulip/master as well as python 3.4.1 / 3.5.
msg215790 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-04-09 01:04
Updated patch (close-3.patch) combining create_connection_close.patch  and close2.patch and adding unit tests. I also improved the create_connection() test to raise the TimeoutError on connect() (loop.sock_connect), not on sock.set_blocking().
msg219307 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-05-28 21:47
Can someone review close-3.patch please?
msg219725 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-06-03 22:16
New changeset d0dd3eb5b5ef by Victor Stinner in branch '3.4':
Issue #21119: asyncio now closes sockets on errors
http://hg.python.org/cpython/rev/d0dd3eb5b5ef

New changeset bbd773ed9584 by Victor Stinner in branch '3.4':
Issue #21119: asyncio: Make sure that socketpair() close sockets on error
http://hg.python.org/cpython/rev/bbd773ed9584

New changeset 8b40483d9a08 by Victor Stinner in branch 'default':
Merge 3.4: Issue #21119, fix ResourceWarning in asyncio
http://hg.python.org/cpython/rev/8b40483d9a08
msg219726 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-03 22:21
I fixed the issues in Tulip, Python 3.4 and 3.5. Thanks for the report.
History
Date User Action Args
2014-06-03 22:21:17vstinnersetstatus: open -> closed
resolution: works for me -> fixed
messages: + msg219726
2014-06-03 22:16:54python-devsetnosy: + python-dev
messages: + msg219725
2014-05-28 21:47:53vstinnersetmessages: + msg219307
2014-04-09 01:04:55vstinnersetfiles: + close-3.patch

messages: + msg215790
2014-04-07 00:52:04landerssonsetresolution: works for me
messages: + msg215674
2014-04-04 16:52:08vstinnersetfiles: + close2.patch

messages: + msg215535
2014-04-04 16:39:26vstinnersetfiles: + create_connection_close.patch
versions: + Python 3.5
nosy: + yselivanov, gvanrossum, vstinner

messages: + msg215534

keywords: + patch
2014-04-01 03:04:44landerssoncreate