classification
Title: Support of 'with' statement fo TarFile class
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: lars.gustaebel Nosy List: brian.curtin, eric.araujo, ezio.melotti, jaime.buelta, lars.gustaebel, meador.inge, r.david.murray
Priority: normal Keywords: needs review, patch

Created on 2009-10-29 07:29 by jaime.buelta, last changed 2010-03-03 12:16 by lars.gustaebel. This issue is now closed.

Files
File name Uploaded Description Edit
tarfileWithSupport.patch jaime.buelta, 2009-10-29 07:29 tarfile with statement support patch
tarfileWithSupportv2.patch r.david.murray, 2010-02-22 03:22
issue7232.diff brian.curtin, 2010-02-22 15:59 minimal patch with test and doc
issue7232.2.diff meador.inge, 2010-02-25 04:06
issue7232.3.diff lars.gustaebel, 2010-02-27 11:37
issue7232.4.diff brian.curtin, 2010-02-27 22:24 use assertRaises as a context manager
issue7232.5.diff brian.curtin, 2010-02-28 01:21
issue7232.6.diff lars.gustaebel, 2010-02-28 09:02
issue7232.7.diff meador.inge, 2010-02-28 17:08
issue7232.8.diff lars.gustaebel, 2010-02-28 19:02
issue7232.9.diff lars.gustaebel, 2010-02-28 21:05
Messages (18)
msg94645 - (view) Author: Jaime Buelta (jaime.buelta) Date: 2009-10-29 07:29
Currently, the TarFile is not supporting the 'with' statement, which I
think it should for coherence with other file classes. 

I've already created a patch including it for consideration.
msg94646 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2009-10-29 08:10
Please clean up the patch, and I take another look at it.
msg94647 - (view) Author: Jaime Buelta (jaime.buelta) Date: 2009-10-29 08:24
I've cleaned the patch, I don't now why Eclipse added a lot of garbage,
sorry.
msg99698 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-02-22 01:39
Here is a patch which expands on Jaime's patch. I was converting tests for #7944 and looked at test_tarfile, and implemented the same feature that he did.

All places where context managers should be used in the test, they are used. Includes a doc update with a small example.
msg99707 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-02-22 03:22
Woops, I accidentally deleted one of the patch files.  Adding back.
msg99761 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-02-22 15:59
The last patch does more than it should for this issue. Here is a minimal patch with the change, test, and doc updates.
msg100083 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2010-02-25 04:06
Built on Brian's patch by adding the following items:

   * Added a unit test case to cover exceptional conditions.
   * Added doc strings on __enter__ and __exit__ (more consistent
     with the surrounding code).
   * Spelling error in doc update: s/manaager/manager/.
   * Link doc update to context manager type documentation (just in
     case the tarfile user is unfamiliar with context manager types).
msg100181 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-02-27 11:37
I have taken the most recent patch by Meador, fixed the documentation example and made the test more verbose. I will check in my patch on time before 2.7 alpha 4 is released if no one of you protests.
msg100193 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-02-27 22:24
What about changing the exception test to something like what I did in issue7232.4.diff?
msg100195 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2010-02-28 00:45
> What about changing the exception test to something like what I did in > issue7232.4.diff?

That is definitely more succinct, but Lars' solution provides more information about _why_ the test fails.  IMHO, the descriptiveness is
more important than succinctness.  Especially when debugging a failed
test.
msg100198 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-02-28 01:21
Good point. How about version 5? It uses base Exception in the context manager, which will allow us to differentiate between no exception being raised, and the wrong one being raised. After the context manager, we check the type of the exception to make sure it's correct.

I changed the exception being raised to an IOError. It could be anything, but given that AssertionError is what gets raised by the assert functions, it seemed better to avoid that one specifically. This is fairly minor, though.

If you change the "raise IOError" to be a pass statement, or another type of exception, you can see that the same level of information is given to you as in patch version 3 by Lars.
msg100199 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-02-28 05:27
Hello

Minor note: I think magic methods shouldn’t have docstrings, because their name is enough doc (or at least enough to go read the doc). At most a one-line comment like “context protocol” can be useful. (The exception is __init__, which doesn’t have a defined set of arguments.)

Cheers
msg100201 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-02-28 09:02
Another version of the patch (issue7232.6.diff) that checks if the TarFile object is still open in the __enter__() method (plus a test for that). I removed the docstrings as Eric suggested. This is common practice in the standard library.
msg100210 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2010-02-28 17:08
> This is common practice in the standard library.

This doesn't necessarily mean it is a correct practice :-).  All 
kidding aside, I think the assumption that the standard documentation 
on '__enter__' and '__exit__' is sufficient is a bad one.  With respect 
to how the 'tarfile' versions of these methods behave, that 
documentation is not that helpful.

In particular, the special behavior of an 'IOError' potentially being 
thrown from '__enter__' and the fact that '__exit__' does not swallow 
the exception.  These special behaviors should be documented either in 
a docstring or the library documentation.  I think this is important, 
but I may be being a bit pedantic.

Also, the last change to 'test_context_manager_exception' has a bug. 
If the call to 'tarfile.open' throws an exception, then the call to
'self.assertRaises' will swallow it.  This will cause an undefined
variable reference to 'tar' in 'self.assertTrue(tar.closed, ...)'.  I 
attached another update that fixes this problem.
msg100216 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-02-28 19:02
IMO it is okay for __enter__() and __exit__() not to have docstrings.
I cannot see what's so special about the behaviour of __enter__() and __exit__().

__enter__() raises IOError only if the TarFile object has been already closed. This is exactly the behaviour I would expect, because it is the same every other TarFile method does when the object has been closed. IOW, using a closed TarFile as a context manager is the programmer's mistake, and I don't feel the need to document that case.

The fact that __exit__() only closes the TarFile object and does not swallow exceptions is what everyone expects from a "file object". It is the only logical thing to do, no need to document that either.

The test_context_manager_exception() test is fine. If the call to tarfile.open() really raises an exception then something is so terribly wrong and probably all of the testsuite's 200 tests will fail anyway. We can safely assume here that this will work, no need to double-check.

However, I have changed the docs again to be a bit more specific.
msg100218 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2010-02-28 19:22
Patch 'issue7232.8.diff' looks good to me.  Thanks for the explanation.
msg100222 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-02-28 21:05
I found an issue that needs to be addressed: if there is an error while the TarFile object is opened for writing, we cannot simply call TarFile.close() in the __exit__() method. close() would try to finalize the archive, i.e. write two zero end-of-archive blocks and a number of padding blocks.

I changed __exit__() to call close() only if everything went fine. If there was an exception only the most basic cleanup is done.

I added more tests and adapted the docs.
msg100339 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-03-03 12:15
Okay, it is done, see r78623 (trunk) and r78626 (py3k).

Thanks to all for your work and support!
History
Date User Action Args
2010-03-03 12:16:02lars.gustaebelsetstage: patch review -> resolved
2010-03-03 12:15:01lars.gustaebelsetstatus: open -> closed
resolution: accepted
messages: + msg100339
2010-02-28 21:05:25lars.gustaebelsetfiles: + issue7232.9.diff

messages: + msg100222
2010-02-28 19:22:40meador.ingesetmessages: + msg100218
2010-02-28 19:02:37lars.gustaebelsetfiles: + issue7232.8.diff

messages: + msg100216
2010-02-28 17:08:41meador.ingesetfiles: + issue7232.7.diff

messages: + msg100210
2010-02-28 09:02:15lars.gustaebelsetfiles: + issue7232.6.diff

messages: + msg100201
2010-02-28 05:27:51eric.araujosetnosy: + eric.araujo
messages: + msg100199
2010-02-28 01:21:07brian.curtinsetfiles: + issue7232.5.diff

messages: + msg100198
2010-02-28 00:45:53meador.ingesetmessages: + msg100195
2010-02-27 22:24:50brian.curtinsetfiles: + issue7232.4.diff

messages: + msg100193
2010-02-27 11:37:02lars.gustaebelsetfiles: + issue7232.3.diff

messages: + msg100181
2010-02-25 04:06:31meador.ingesetfiles: + issue7232.2.diff
nosy: + meador.inge
messages: + msg100083

2010-02-22 15:59:20brian.curtinsetfiles: + issue7232.diff

messages: + msg99761
2010-02-22 15:57:42brian.curtinsetfiles: - issue7944_tarfile.diff
2010-02-22 03:22:24r.david.murraysetfiles: + tarfileWithSupportv2.patch
nosy: + r.david.murray
messages: + msg99707

2010-02-22 02:56:18r.david.murraysetfiles: - tarfileWithSupportv2.patch
2010-02-22 01:40:21brian.curtinlinkissue7944 dependencies
2010-02-22 01:39:47brian.curtinsetfiles: + issue7944_tarfile.diff

nosy: + brian.curtin
messages: + msg99698

keywords: + needs review
2009-10-30 01:13:44ezio.melottisetpriority: normal
nosy: + ezio.melotti
versions: + Python 3.2

stage: patch review
2009-10-29 08:24:38jaime.bueltasetfiles: + tarfileWithSupportv2.patch

messages: + msg94647
2009-10-29 08:10:52lars.gustaebelsetassignee: lars.gustaebel

messages: + msg94646
nosy: + lars.gustaebel
2009-10-29 07:29:33jaime.bueltacreate