This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: urllib hangs when closing connection
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.2, Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: giampaolo.rodola Nosy List: daniel.ugra, giampaolo.rodola, orsenthil, python-dev, r.david.murray
Priority: normal Keywords: 3.3regression, patch

Created on 2012-10-17 21:38 by daniel.ugra, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
urllib.patch giampaolo.rodola, 2012-10-18 01:03
Pull Requests
URL Status Linked Edit
PR 13951 open danh, 2019-06-10 23:44
Messages (9)
msg173209 - (view) Author: Ugra Dániel (daniel.ugra) Date: 2012-10-17 21:38
With version 3.3 (64 bit on Arch Linux) Python now hangs when closing FTP connection on some sites:

url = "ftp://ftp.fu-berlin.de/pub/misc/movies/database/ratings.list.gz"

with urllib.request.urlopen( url ):
	pass
msg173212 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-17 21:50
I guess this is about urllib, not ftplib.
msg173223 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-10-18 00:42
Well, urllib calls ftplib for ftp urls, so it might be about ftplib.  But in that case it would be better to have a reproducer that *just* uses ftplib.
msg173224 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2012-10-18 01:03
I can reproduce the issue on python 3.3.
To have a clearer understanding on what's going on use:

import urllib.request
import ftplib
ftplib.FTP.debugging = 4
url = "ftp://ftp.fu-berlin.de/pub/misc/movies/database/ratings.list.gz"
with urllib.request.urlopen(url):
    pass

The problem appears to be endtransfer() erroneously calling ftp.voidresp() (it just shouldn't):
http://hg.python.org/cpython/file/f6fcff683866/Lib/urllib/request.py#l2328
Patch in attachment fixes the issue.
msg173323 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-19 11:25
New changeset edeeb727df86 by Giampaolo Rodola' in branch 'default':
Fix issue #16270: urllib may hang when used for retrieving files via FTP by using a context manager.
http://hg.python.org/cpython/rev/edeeb727df86
msg173325 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-19 11:35
New changeset 2e6bcbb0ff59 by Giampaolo Rodola' in branch '3.3':
Fix issue #16270: urllib may hang when used for retrieving files via FTP by using a context manager.
http://hg.python.org/cpython/rev/2e6bcbb0ff59
msg173326 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-10-19 11:40
New changeset 7f0d9637a3ad by Giampaolo Rodola' in branch '3.2':
Fix issue #16270: urllib may hang when used for retrieving files via FTP by using a context manager.
http://hg.python.org/cpython/rev/7f0d9637a3ad
msg265227 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-05-10 08:13
New changeset 44d02a5d59fb by Senthil Kumaran in branch '2.7':
Closes issue26960.
https://hg.python.org/cpython/rev/44d02a5d59fb
msg286020 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2017-01-22 17:46
The original problem here was retrieving files failed (hung) when it was tried via context-manager

1. For e.g after "reverting" the changes, if you use the original file without the context manager, it will succeed.

fobj = urllib.request.urlopen( url )

That indicates the fix in changing something in endtransfer method was improper. I have given more information here: http://bugs.python.org/issue27973#msg286016

The proper fix will be revert the endtransfer changes, but fix it in contextmanager scenario. Add test cases to cover all these scenarios.
History
Date User Action Args
2022-04-11 14:57:37adminsetgithub: 60474
2019-06-10 23:44:24danhsetpull_requests: + pull_request13820
2017-01-22 17:46:13orsenthilsetmessages: + msg286020
2016-05-10 08:13:07python-devsetmessages: + msg265227
2012-10-19 11:41:53giampaolo.rodolasetstatus: open -> closed
assignee: giampaolo.rodola
resolution: fixed
versions: + Python 3.2
2012-10-19 11:40:37python-devsetmessages: + msg173326
2012-10-19 11:35:08python-devsetmessages: + msg173325
2012-10-19 11:25:28python-devsetnosy: + python-dev
messages: + msg173323
2012-10-18 01:07:46giampaolo.rodolasettitle: ftplib hangs when closing connection -> urllib hangs when closing connection
2012-10-18 01:03:59giampaolo.rodolasetfiles: + urllib.patch
keywords: + patch
messages: + msg173224
2012-10-18 00:42:54r.david.murraysetnosy: + r.david.murray
messages: + msg173223
2012-10-17 21:50:01giampaolo.rodolasetmessages: + msg173212
2012-10-17 21:48:48pitrousetkeywords: + 3.3regression
nosy: + orsenthil, giampaolo.rodola

versions: + Python 3.4
2012-10-17 21:38:49daniel.ugracreate