classification
Title: 2.7rc1 tarfile.py: `bltn_open(targetpath, "wb")` -> IOError: Is a directory
Type: crash Stage:
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: lars.gustaebel Nosy List: lars.gustaebel, srid, vstinner
Priority: normal Keywords:

Created on 2010-06-09 21:20 by srid, last changed 2010-06-14 17:56 by srid. This issue is now closed.

Messages (7)
msg107419 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-06-09 21:20
1. Find an OSX 10.5.8 machine 
2. wget http://hntool.googlecode.com/files/hntool-0.1.1.tar.gz
3. $ python2.7 -c "import tarfile as T; t=T.open('hntool-0.1.1.tar.gz'); t.extractall()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 2046, in extractall
    self.extract(tarinfo, path)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 2083, in extract
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 2159, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py", line 2198, in makefile
    target = bltn_open(targetpath, "wb")
IOError: [Errno 21] Is a directory: './hntool-0.1.1/hntool'

...


Note that, when extracted open via other tools, hntool-0.1.1.tar.gz contains within it ... both a directory named "HnTool" (note the case) and a file  called "hntool".
msg107479 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-06-10 20:06
Unfortunately I do not have access to an OS X machine. Is this problem specific to 2.7rc1 or are other versions affected as well? I thought the OS X filesystem was case sensitive ...
msg107493 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-06-10 21:29
On 2010-06-10, at 1:06 PM, Lars Gustäbel wrote:

> Is this problem specific to 2.7rc1

Yes.

> or are other versions affected as well?

Nope, at least ... I know that 2.6 doesn't have this problem.
msg107540 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-06-11 10:52
I found the problem. As of r76780 the default for the TarFile.errorlevel argument changed from 0 (suppress errors and write them to the debug log instead) to 1 (raise exceptions for fatal extraction errors). This change was not backported to the 2.6 branch back then (it was blocked in r76781).

This means, that Python 2.6 does not succeed either, but the error is simply suppressed.

Ergo, this is no regression but a simple filesystem issue.
msg107798 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-06-14 16:45
I am curious as to why this should still "fail" as the OSX filesystem is case sensitive. Finder has no problems with extrating this particular tarball. Do you think this is a (separate) bug, or is this by design (why?)?

If this is by design, does it make sense to raise a custom exception from TarError, instead of the generic IOError?
msg107803 - (view) Author: Lars Gustäbel (lars.gustaebel) * (Python committer) Date: 2010-06-14 17:42
a) The point is: the operation simply wouldn't fail on a case-sensitive filesystem. There is no platform-specific or otherwise special code in TarFile.makefile(). It simply tries to extract the file and the filesystem layer says no, because it believes there is already a directory with the same name. The same thing happens on a Windows filesystem BTW. The problem boils down to this:

>>> os.mkdir("A")
>>> open("a", "w")
IOError: [Errno 21] Is a directory: 'a'

And IIUC, the Mac OS X filesystem is case-preserving(!) by default, with the possibility to create new filesystems as case-sensitive. As I said, my Mac OS X expertise is almost nonexistent, you might as well ask someone with more knowledge on Python on Mac OS X.

b) I don't know what Finder does with that archive, but I cannot think of any other way than either not to extract the file at all or to extract it under a different name or to remove the directory first and then extract the file. Could you please examine how Finder extracts this archive?

c) IMHO the IOError exception is perfectly fine, because this kind of issue is outside of tarfile's scope. We hit a filesystem limit here. Also, there is no decent way to work around this problem, and I think there is no need to either.
msg107805 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-06-14 17:56
On 2010-06-14, at 10:43 AM, Lars Gustäbel wrote:

> Lars Gustäbel <lars@gustaebel.de> added the comment:
> 
> a) The point is: the operation simply wouldn't fail on a case-sensitive filesystem. There is no platform-specific or otherwise special code in TarFile.makefile(). It simply tries to extract the file and the filesystem layer says no, because it believes there is already a directory with the same name. The same thing happens on a Windows filesystem BTW. The problem boils down to this:
> 
>>>> os.mkdir("A")
>>>> open("a", "w")
> IOError: [Errno 21] Is a directory: 'a'
> 
> And IIUC, the Mac OS X filesystem is case-preserving(!) by default, with the possibility to create new filesystems as case-sensitive. As I said, my Mac OS X expertise is almost nonexistent, you might as well ask someone with more knowledge on Python on Mac OS X.

Ah, I see. I didn't know about this compatibility quirk.

> b) I don't know what Finder does with that archive, but I cannot think of any other way than either not to extract the file at all or to extract it under a different name or to remove the directory first and then extract the file. Could you please examine how Finder extracts this archive?

I was wrong about Finder extracting the tarball; actually I used the Finder on my macbook (10.6), where `open("a", "w")` succeeds. Whereas this issue seems to be reproducible in 10.5 or 10.4 only.

I did try to use `tar` this time, and it failed:

$ tar zxf hntool-0.1.1.tar.gz 
tar: hntool-0.1.1/hntool: Cannot open: File exists
tar: Error exit delayed from previous errors

> c) IMHO the IOError exception is perfectly fine, because this kind of issue is outside of tarfile's scope. We hit a filesystem limit here. Also, there is no decent way to work around this problem, and I think there is no need to either.

In this case, yes ... I agree.
History
Date User Action Args
2010-06-14 17:56:44sridsetmessages: + msg107805
2010-06-14 17:42:59lars.gustaebelsetmessages: + msg107803
2010-06-14 16:45:32sridsetmessages: + msg107798
2010-06-11 10:52:58lars.gustaebelsetstatus: open -> closed
resolution: wont fix
messages: + msg107540
2010-06-10 21:31:15vstinnersetnosy: + vstinner
2010-06-10 21:29:26sridsetmessages: + msg107493
2010-06-10 20:06:32lars.gustaebelsetassignee: lars.gustaebel
2010-06-10 20:06:12lars.gustaebelsetnosy: + lars.gustaebel
messages: + msg107479
2010-06-09 21:20:53sridcreate