classification
Title: tarfile.extractall on existing symlink in Ubuntu overwrites target file, not symlink, unlinke GNU tar
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: duplicate
Dependencies: Superseder: The tarfile module crashes when tarfile contains a symlink and unpack directory contain it too
View: 23228
Assigned To: Nosy List: lars.gustaebel, martin.panter, michael.brandl@aid-driving.eu
Priority: normal Keywords:

Created on 2018-12-13 15:22 by michael.brandl@aid-driving.eu, last changed 2019-01-06 09:58 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
symLinkBugRepro.tar.gz michael.brandl@aid-driving.eu, 2018-12-13 15:22 Zip file containing a bash script and a python script for repro of tarfile symlink issue.
Messages (3)
msg331762 - (view) Author: Michael Brandl (michael.brandl@aid-driving.eu) Date: 2018-12-13 15:22
In Ubuntu 16.04, with python 3.5, as well as custom built 3.6 and 3.7.1:

Given a file foo.txt (with content "foo") and a symlink myLink to it, packed in a tar,  and   a file bar.txt (with content "bar") with a symlink myLink to it, packed in another tar,
unpacking the two tars into the same folder (first foo.tar, then bar.tar) leads to the following behavior:

In GNU tar, the directory will contain:
foo.txt (content "foo")
bar.txt (content "bar")
myLink ->bar.txt.

Using python's tarfile however, the result of calling tarfile.extractall on the two tars will give:
foo.txt (content "bar")
bar.txt (content "bar")
myLink ->foo.txt.


Repro: 
1. Unpack the attached symLinkBugRepro.tar.gz into a new folder
2. run > bash repoSymlink.bash (does exactly what is described above)
3. if the last two lines of the output are "bar" and "bar" (instead of "foo" and "bar"), then the content of foo.txt has been overwritten.

Note that this is related to issues like
https://bugs.python.org/issue23228
https://bugs.python.org/issue1167128
https://bugs.python.org/issue19974
https://bugs.python.org/issue10761

None of these issues target the issue at hand, however.

The problem lies in line 2201 of https://github.com/python/cpython/blob/master/Lib/tarfile.py:
The assumption is that any exception only comes from the os not supporting symlinks. But here, the exception comes from the symlink already existing, which should be caught separately. The correct behavior is then NOT to extract the member, but rather to overwrite the symlink (as GNU tar does).
msg331913 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2018-12-16 01:45
The first aspect, incorrectly assuming the OS does not support symlinks, is described at <https://bugs.python.org/issue23228#msg265146>. Lars proposed a fix <https://bugs.python.org/file42780/windowserror.diff> which will let the OS exception escape to the caller. However I think that patch needs more work.

The second aspect is replacing existing symlinks and other directory entries. This was implemented in 2.7 in Issue 10761 and Issue 12088 (only when replacing non-subdirectories with symbolic links and hard links), and is discussed more generally in Issue 19974.

I suggest to close this in favour of resolving Issue 23228 and Issue 19974.
msg331953 - (view) Author: Michael Brandl (michael.brandl@aid-driving.eu) Date: 2018-12-17 09:02
Sounds good to me.
History
Date User Action Args
2019-01-06 09:58:26martin.pantersetstatus: open -> closed
resolution: duplicate
stage: resolved
2018-12-17 09:02:06michael.brandl@aid-driving.eusetmessages: + msg331953
2018-12-16 01:45:59martin.pantersetnosy: + lars.gustaebel, martin.panter
superseder: The tarfile module crashes when tarfile contains a symlink and unpack directory contain it too
messages: + msg331913
2018-12-13 15:22:39michael.brandl@aid-driving.eucreate