This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: urllib.URLopener prematurely deletes files on cleanup
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: carlbook, iritkatriel, orsenthil
Priority: normal Keywords:

Created on 2011-07-25 15:34 by carlbook, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bug2.py carlbook, 2011-07-25 15:34 urllib exception
bug3.py carlbook, 2011-07-25 16:57
Messages (6)
msg141094 - (view) Author: Carl (carlbook) Date: 2011-07-25 15:34
urllib.URLopener (or urllib.request.URLopener for Python 3) and user defined classes that inherit from these prematurely delete files upon cleanup.  Any temporary files downloaded using the .retrieve() method are deleted when an instance of a URLopener is garbage collected.

I feel this is a violation since the filename is returned to the caller and then silently deleted.  It is possible to simply override the .cleanup() method, but I feel this is not a good solution.
msg141095 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-07-25 15:47
urlretrieve is a helper function from urllib  module. The way to use
it is:

('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
>>> import os
>>> os.stat('/tmp/tmpe873xe')
posix.stat_result(st_mode=33152, st_ino=4462517, st_dev=2054,
st_nlink=1, st_uid=1000, st_gid=1000, st_size=33128,
st_atime=1311608669, st_mtime=1311608670, st_ctime=1311608670)

Works properly for me 3.3 and 2.7. Is there any service that is
running on your machine that deleting the tmp files as soon as they
are created?
msg141096 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-07-25 15:51
urlretrieve is a helper function from urllib  module. The way to use
it is:

('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
>>> import os
>>> os.stat('/tmp/tmpe873xe')

Works!
msg141097 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2011-07-25 15:54
Tracker stripped off the code. Here is it.

import urllib.request
urllib.request.urlretrieve('http://bugs.python.org')
('/tmp/tmpe873xe', <http.client.HTTPMessage object at 0xb72c2f6c>)
import os
os.stat('/tmp/tmpe873xe')
posix.stat_result(st_mode=33152, st_ino=4462517, st_dev=2054, st_nlink=1, st_uid=1000, st_gid=1000, st_size=33128, st_atime=1311608669, st_mtime=1311608670, st_ctime=1311608670)
msg141102 - (view) Author: Carl (carlbook) Date: 2011-07-25 16:57
@orsenthil, that is the correct behavior if you do not want to override any of URLopener's handlers for error codes.  In my case, I wanted to override FancyURLopener (a child class of URLopener) to override HTTP 401 behavior.  Using urlretrieve is not correct in this case.

Also included python 3.2 code, I didn't test 3.1.
msg389548 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-03-26 12:27
I think this is out of date as this feature was deprecated in issue10050.
History
Date User Action Args
2022-04-11 14:57:20adminsetgithub: 56847
2021-05-08 11:00:20iritkatrielsetresolution: out of date
2021-05-08 11:00:12iritkatrielsetstatus: pending -> closed
stage: resolved
2021-03-26 12:27:57iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg389548

components: + Library (Lib), - None
2011-07-25 16:57:50carlbooksetfiles: + bug3.py

messages: + msg141102
2011-07-25 15:54:38orsenthilsetmessages: + msg141097
2011-07-25 15:51:33orsenthilsetmessages: + msg141096
2011-07-25 15:47:23orsenthilsetnosy: + orsenthil
messages: + msg141095
2011-07-25 15:34:57carlbookcreate