classification
Title: Add context manager for temporary directory
Type: enhancement Stage: patch review
Components: Versions: Python 3.2
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: daniel.ugra, ezio.melotti, georg.brandl, giampaolo.rodola, nascheme, ncoghlan, pitrou
Priority: normal Keywords: patch

Created on 2009-02-07 17:54 by nascheme, last changed 2011-09-20 10:32 by daniel.ugra. This issue is now closed.

Files
File name Uploaded Description Edit
tempdir.patch nascheme, 2009-02-07 17:54 review
tempdir2.patch nascheme, 2009-02-14 06:39 review
Messages (14)
msg81342 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-07 17:54
I noticed that it would be nice to have a temporary directory context
manager while trying to fix a broken unittest.  The attached patch
provides a pretty minimal implementation.  There appears to be lots of
unit tests that could use such a thing (just search for "rmtree").

I'm not sure if TemporaryDirectory is the best name.  Also, perhaps it
should have a __del__ method although that's tricky business.
msg81349 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2009-02-07 21:31
A __del__ method is definitely desirable (tempfile._TemporaryFileWrapper
gives an example of how to cache the relevant globals on the class
object to avoid attempting calls to None during interpreter shutdown).

The new examples are good, but may give the misleading impression that
TemporaryFile can't be used as a context manager (it can, and the file
will be closed at the end of the with statement). A second file example
showing it being used as a context manager would probably be helpful.

There is also at least one existing contextlib based temp_dir context
managers that could be replaced given the addition of this (i.e. in
test_cmd_line_script.py, "test_dir" could be changed to a class that
inherits from tempfile.TemporaryDirectory and overrides __enter__ to
invoke realname() on the directory name).
msg81995 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-14 06:39
New version of the patch.  Added a __del__ method that hopefully works
reliably.  Added NEWS and an example of TemporaryFile as a context
manager.  I did not change more tests to use TemporaryDirectory since I
understand hit-and-run modernization of code is generally discouraged. 
That said, I think there are some tests that could have their
reliability improved by using TemporaryDirectory (I started on this
patch when I saw some build bot failures).  I'll look into that later.
msg82044 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 13:23
If you look at the code for os.path.isdir, it uses other globals such as
os.stat, so you might want to reinject them in your class as well...
msg82085 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2009-02-14 16:30
Argh, it's like pulling a sweater thread.  The stat.S_ISDIR function
uses the S_IFDIR global so I would have write my own.  This does not
look like good maintainable code, IMO.

Maybe there should be a special flag modules can set to make them get
cleared later in the interpreter cleanup procedure.  Modules could set
it if they don't create reference cycles themselves and don't hold
references to user objects.
msg82088 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 16:44
> Maybe there should be a special flag modules can set to make them get
> cleared later in the interpreter cleanup procedure.  Modules could set
> it if they don't create reference cycles themselves and don't hold
> references to user objects.

Or, perhaps we could remove the assignment to None and use a
finalization scheme based on the garbage collector instead.
See #812369 (no patch for the above idea unfortunately).
msg84005 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2009-03-23 12:42
Unassigning for the moment - I'm still interested in the idea, but the
referencing-globals-at-shutdown problem means it isn't going to be the
straightforward review-and-commit that I original thought this patch was
going to be.

I'll still try to get to it before the next 3.1 alpha in a couple of
weeks, but I don't want to stop anyone else from looking at it in the
meantime.
msg97706 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-01-13 10:20
Building a collection of issues I want to take a look at for 2.7
msg104435 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-04-28 14:59
Missed the boat for 2.7 I'm afraid. Definitely one to take another look at for 3.2 though.
msg119452 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-10-23 17:33
Ping... soon it's too late for 3.2 too :)
msg119463 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2010-10-23 18:41
Personally I would like to have a unique tempfile.mkdtemp which can be used as both a function and a context manager, as it happens for open() and others.
Not sure how to do that though, unless tempfile.mkdtemp gets turned into a class.
There would be objections against doing that?
msg119509 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-10-24 11:23
Committed (with enhanced tests and a few fixes) in r85818

And credit where it's due to test___all__ for picking up a typo in my adjustment to tempfile.__all__ :)

I created issue #10188 to track the shutdown problem. I'm considering taking out the code that attempts to allow the __del__ method to work at shutdown though, since it isn't actually achieving anything (I wanted a record of it in the main repository, which is why I kept it for the initial checkin).
msg119546 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2010-10-25 12:52
Nick, could you comment about my last proposal?
msg119596 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2010-10-26 09:12
Merging the interfaces for mkdtemp and TemporaryDirectory isn't going to happen.

mkstemp/mkdtemp are for when the user wants to control the lifecycle of the filesystem entries themselves. (Note that context management on a regular file-like object only closes the file, it doesn't delete it from the filesystem). They're also intended as relatively thin wrappers around the corresponding C standard library functionality.

The other objects in tempfile (TemporaryFile, TemporaryDirectory, etc) are for when the user wants the lifecycle of the Python object to correspond with the lifecycle of the underlying filesystem element.

That said, TD itself can be used to create the temporary directory without having to use it as a context manager (the underlying directory is created in __init__, not __enter__).
History
Date User Action Args
2011-09-20 10:32:01daniel.ugrasetnosy: + daniel.ugra
2010-10-26 09:12:20ncoghlansetmessages: + msg119596
2010-10-25 12:52:19giampaolo.rodolasetmessages: + msg119546
2010-10-24 11:24:09ncoghlansetstatus: open -> closed
resolution: accepted
2010-10-24 11:23:56ncoghlansetmessages: + msg119509
2010-10-24 09:21:11ncoghlansetassignee: ncoghlan
2010-10-23 18:41:06giampaolo.rodolasetmessages: + msg119463
2010-10-23 17:33:23georg.brandlsetnosy: + georg.brandl
messages: + msg119452
2010-04-28 20:41:31giampaolo.rodolasetnosy: + giampaolo.rodola
2010-04-28 14:59:25ncoghlansetassignee: ncoghlan -> (no value)
messages: + msg104435
versions: - Python 2.7
2010-01-13 10:20:51ncoghlansetassignee: ncoghlan
messages: + msg97706
versions: + Python 3.2, - Python 3.1
2010-01-13 00:42:39ezio.melottisetnosy: + ezio.melotti
2009-03-23 12:42:21ncoghlansetassignee: ncoghlan -> (no value)
messages: + msg84005
2009-02-14 16:44:59pitrousetmessages: + msg82088
2009-02-14 16:30:29naschemesetmessages: + msg82085
2009-02-14 13:23:08pitrousetnosy: + pitrou
messages: + msg82044
2009-02-14 06:39:47naschemesetfiles: + tempdir2.patch
messages: + msg81995
2009-02-08 03:39:09ncoghlansettitle: Add content manager for temporary directory -> Add context manager for temporary directory
2009-02-08 03:38:39ncoghlansetpriority: normal
versions: + Python 3.1, Python 2.7
2009-02-07 21:31:11ncoghlansetmessages: + msg81349
2009-02-07 17:54:19naschemecreate