classification
Title: urllib2 cannot fully read FTP file
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, iritkatriel, keakon, r.david.murray
Priority: normal Keywords:

Created on 2015-11-16 13:47 by keakon, last changed 2020-11-30 10:21 by iritkatriel. This issue is now closed.

Messages (4)
msg254733 - (view) Author: keakon (keakon) Date: 2015-11-16 13:47
I found the bug from this slide: http://sector.ca/Portals/17/Presentations15/SecTor_Branca.pdf

The second way cannot fully read the file.

import urllib2

url = 'ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest'
response = urllib2.urlopen(url)
data = response.read()
print len(data)  # 6653498

data = urllib2.urlopen(url).read()
print len(data)  # 65536


It might be something wrong with the FTP server. It's OK when I read from my own FTP server.
msg254736 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-16 14:59
Most likely it is a timing issue, with the response object sometimes getting GCed before the read is complete (your local server would make this less likely since the read would complete sooner).

I think this has been fixed in python3.  Can you check?

Note, however, that this is not best practice code.  Ideally you should explicitly close the response object after reading from it.
msg355134 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2019-10-22 15:19
It works as expected on master (3.9.0a0). I think we can close this
msg382139 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-30 10:21
Works for me on master (3.10) too.
History
Date User Action Args
2020-11-30 10:21:26iritkatrielsetstatus: open -> closed

nosy: + iritkatriel
messages: + msg382139

resolution: out of date
stage: resolved
2019-10-22 15:19:17BTaskayasetnosy: + BTaskaya
messages: + msg355134
2015-11-16 14:59:48r.david.murraysetnosy: + r.david.murray
messages: + msg254736
2015-11-16 13:47:11keakoncreate