'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009

adunand · 2011-08-20T22:57:40Z

BPO	12800
Nosy	@gustaebel, @catlee, @taleinat, @serhiy-storchaka, @JulienPalard, @miss-islington, @websurfer5
PRs	bpo-12800: 'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #13217 bpo-12800: 'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #20972 bpo-12800: tarfile: Restore fix from 011525ee9 #21409 [3.9] bpo-12800: tarfile: Restore fix from 011525ee9 (GH-21409) #23508 [3.8] bpo-12800: tarfile: Restore fix from 011525ee9 (GH-21409) #23509
Files	test.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gustaebel'
closed_at = <Date 2020-12-18.18:53:15.392>
created_at = <Date 2011-08-20.22:57:39.951>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library']
title = "'tarfile.StreamError: seeking backwards is not allowed' when extract symlink"
updated_at = <Date 2020-12-18.18:53:15.392>
user = 'https://bugs.python.org/adunand'

bugs.python.org fields:

activity = <Date 2020-12-18.18:53:15.392>
actor = 'mdk'
assignee = 'lars.gustaebel'
closed = True
closed_date = <Date 2020-12-18.18:53:15.392>
closer = 'mdk'
components = ['Library (Lib)']
creation = <Date 2011-08-20.22:57:39.951>
creator = 'adunand'
dependencies = []
files = ['49168']
hgrepos = []
issue_num = 12800
keywords = ['patch']
message_count = 13.0
messages = ['142580', '221577', '221595', '223171', '341969', '369061', '369119', '369120', '373384', '377173', '381803', '381807', '381810']
nosy_count = 9.0
nosy_names = ['lars.gustaebel', 'catlee', 'taleinat', 'adunand', 'serhiy.storchaka', 'andrew.garner', 'mdk', 'miss-islington', 'Jeffrey.Kintscher']
pr_nums = ['13217', '20972', '21409', '23508', '23509']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12800'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

adunand · 2011-08-20T22:57:40Z

When you extractall a tarball containing a symlink in stream mode ('r|'), an Exception happens:

Traceback (most recent call last):
    File "./test_extractall_stream_symlink.py", line 26, in <module>
    tar.extractall(path=destdir)
    File "/usr/lib/python3.2/tarfile.py", line 2134, in extractall
    self.extract(tarinfo, path, set_attrs=not tarinfo.isdir())
    File "/usr/lib/python3.2/tarfile.py", line 2173, in extract
    set_attrs=set_attrs)
    File "/usr/lib/python3.2/tarfile.py", line 2249, in _extract_member
    self.makefile(tarinfo, targetpath)
    File "/usr/lib/python3.2/tarfile.py", line 2289, in makefile
    source.seek(tarinfo.offset_data)
    File "/usr/lib/python3.2/tarfile.py", line 553, in seek
    raise StreamError("seeking backwards is not allowed")
    tarfile.StreamError: seeking backwards is not allowed

You can reproduce the bug with this snippet of code:

TEMPDIR='/tmp/pyton_test'
os.mkdir(TEMPDIR)
tempdir = os.path.join(TEMPDIR, "testsymlinks")
temparchive = os.path.join(TEMPDIR, "testsymlinks.tar")
destdir = os.path.join(TEMPDIR, "extract")
os.mkdir(tempdir)
try:
source_file = os.path.join(tempdir,'source')
target_file = os.path.join(tempdir,'symlink')
with open(source_file,'w') as f:
f.write('something\n')
os.symlink('source', target_file)
tar = tarfile.open(temparchive,'w')
tar.add(target_file, arcname=os.path.basename(target_file))
tar.add(source_file, arcname=os.path.basename(source_file))
tar.close()
fo = open(temparchive, 'rb')
tar = tarfile.open(fileobj=fo, mode='r|')
try:
tar.extractall(path=destdir)
finally:
tar.close()
finally:
os.unlink(temparchive)
shutil.rmtree(TEMPDIR)

If source_file is added before target_file, there is no Exception raised. But it still raised when you create the same tarball with GNU tar.

BreamoreBoy · 2014-06-25T21:55:30Z

ping.

serhiy-storchaka · 2014-06-26T07:31:32Z

All works to me without exception in 2.7, 3.3 and 3.4.

andrewgarner · 2014-07-16T04:53:35Z

This seems to be a similar to bpo-10761 where symlinks are not being overwritten by TarFile.extract but is only an issue in streaming mode and only in python3. To reproduce, attempt to extract a symlink from a tarfile opened with 'r|' and overwrite an existing file.

Here's a simple scripts that demonstrates this behavior adapted from Aurélien's.

#!/usr/bin/python

import os
import shutil
import sys
import tempfile
import tarfile


def main():
    tmpdir = tempfile.mkdtemp()
    try:
        os.chdir(tmpdir)
        source = 'source'
        link = 'link'
        temparchive = 'issue12800'
        # create source
        with open(source, 'wb'):
            pass
        os.symlink(source, link)
        with tarfile.open(temparchive, 'w') as tar:
            tar.add(source, arcname=os.path.basename(source))
            tar.add(link, arcname=os.path.basename(link))

        with open(temparchive, 'rb') as fileobj:
            with tarfile.open(fileobj=fileobj, mode='r|') as tar:
                tar.extractall(path=tmpdir)
    finally:
        shutil.rmtree(tmpdir)

if __name__ == '__main__':
    sys.exit(main())

On python 3.3.2 I get the following results:

$ python3.3 issue12800.py
Traceback (most recent call last):
  File "issue12800.py", line 32, in <module>
    sys.exit(main())
  File "issue12800.py", line 27, in main
    tar.extractall(path=tmpdir)
  File "/usr/lib64/python3.3/tarfile.py", line 1984, in extractall
    self.extract(tarinfo, path, set_attrs=not tarinfo.isdir())
  File "/usr/lib64/python3.3/tarfile.py", line 2023, in extract
    set_attrs=set_attrs)
  File "/usr/lib64/python3.3/tarfile.py", line 2100, in _extract_member
    self.makelink(tarinfo, targetpath)
  File "/usr/lib64/python3.3/tarfile.py", line 2181, in makelink
    os.symlink(tarinfo.linkname, targetpath)
FileExistsError: [Errno 17] File exists: '/tmp/tmpt0u1pn/link'

On python 3.4.1 I get the following results:

$ python3.4 issue12800.py
Traceback (most recent call last):
  File "/usr/lib64/python3.4/tarfile.py", line 2176, in makelink
    os.symlink(tarinfo.linkname, targetpath)
FileExistsError: [Errno 17] File exists: 'source' -> '/tmp/tmp3b96k5f0/link'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "issue12800.py", line 32, in <module>
    sys.exit(main())
  File "issue12800.py", line 27, in main
    tar.extractall(path=tmpdir)
  File "/usr/lib64/python3.4/tarfile.py", line 1979, in extractall
    self.extract(tarinfo, path, set_attrs=not tarinfo.isdir())
  File "/usr/lib64/python3.4/tarfile.py", line 2018, in extract
    set_attrs=set_attrs)
  File "/usr/lib64/python3.4/tarfile.py", line 2095, in _extract_member
    self.makelink(tarinfo, targetpath)
  File "/usr/lib64/python3.4/tarfile.py", line 2187, in makelink
    targetpath)
  File "/usr/lib64/python3.4/tarfile.py", line 2087, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/usr/lib64/python3.4/tarfile.py", line 2126, in makefile
    source.seek(tarinfo.offset_data)
  File "/usr/lib64/python3.4/tarfile.py", line 518, in seek
    raise StreamError("seeking backwards is not allowed")
tarfile.StreamError: seeking backwards is not allowed

websurfer5 · 2019-05-09T06:00:24Z

The problem is in TarFile.makelink() in Lib/tarfile.py. It calls os.symlink() to create the link, which fails because the link already exists and triggers the exception handler. The exception handler then tries to create the linked file under the assumption (per source code comments) that the link creation failed because the system doesn't support symbolic links. The file creation then fails because it requires seeking backwards in the archive.

catlee · 2020-05-16T18:39:10Z

Is there anything I can do to help get this landed? The PR in github works for me.

JulienPalard · 2020-05-17T13:23:16Z

Hi Chris, which exception did you got exactly? Was it caused by the r| mode or by a symlink (or file) already existing?

catlee · 2020-05-17T13:38:02Z

It's caused by the combination of the symlink existing, and having the tarfile opened in r| mode.

If I run the attached test file in a fresh directory, I get the following exception:

raceback (most recent call last):
File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2227, in makelink
os.symlink(tarinfo.linkname, targetpath)
FileExistsError: [Errno 17] File exists: 'message.txt' -> './symlink.txt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../test.py", line 12, in <module>
    tf.extractall()
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2024, in extractall
    self.extract(tarinfo, path, set_attrs=not tarinfo.isdir(),
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2065, in extract
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2145, in _extract_member
    self.makelink(tarinfo, targetpath)
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2237, in makelink
    self._extract_member(self._find_link_target(tarinfo),
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2137, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 2176, in makefile
    source.seek(tarinfo.offset_data)
  File "/home/catlee/.pyenv/versions/3.8.2/lib/python3.8/tarfile.py", line 513, in seek
    raise StreamError("seeking backwards is not allowed")
tarfile.StreamError: seeking backwards is not allowed

JulienPalard · 2020-07-09T09:29:15Z

Strange fact, this was already fixed in 011525e (which closes bpo-10761, nice spot Andrew) but was lost during a merge in 0d28a61:

$ git show 0d28a61d23
commit 0d28a61d233c02c458c8b4a25613be2f4979331e
Merge: ed3a303548 d7c9d9cdcd

$ git show 0d28a61d23:Lib/tarfile.py | grep unlink  # The merge commit does no longer contains the fix

$ git show ed3a303548:Lib/tarfile.py | grep unlink  # The "left" parent does not contains it neither

$ git show d7c9d9cdcd:Lib/tarfile.py | grep unlink  # The "right" one does contains it.
                    os.unlink(targetpath)
                        os.unlink(targetpath)

Stranger fact, the test was not lost during the merge, and still lives today (test_extractall_symlinks).

Happen that the current test is passing because it's in part erroneous, instead of trying to create a symlink on an existing one, it creates a symlink far far away:

(Pdb) p targetpath
'/home/mdk/clones/python/cpython/@test_648875_tmp-tardir/testsymlinks/home/mdk/clones/python/cpython/@test_648875_tmp-tardir/testsymlinks/symlink'

Aditionally it passes anway because tar.errorlevel equals 1, which means the error is logged but not raised.

With the following small patch:

--- a/Lib/test/test_tarfile.py
+++ b/Lib/test/test_tarfile.py
@@ -1339,10 +1339,10 @@ class WriteTest(WriteTestBase, unittest.TestCase):
                 f.write('something\n')
             os.symlink(source_file, target_file)
             with tarfile.open(temparchive, 'w') as tar:
-                tar.add(source_file)
-                tar.add(target_file)
+                tar.add(source_file, arcname="source")
+                tar.add(target_file, arcname="symlink")
             # Let's extract it to the location which contains the symlink
-            with tarfile.open(temparchive) as tar:
+            with tarfile.open(temparchive, errorlevel=2) as tar:
                 # this should not raise OSError: [Errno 17] File exists
                 try:
                     tar.extractall(path=tempdir)

the error is raised as expected: FileExistsError: [Errno 17] File exists: '/home/mdk/clones/python/cpython/@test_649794_tmpæ-tardir/testsymlinks/source' -> '/home/mdk/clones/python/cpython/@test_649794_tmpæ-tardir/testsymlinks/symlink'

I'm opening an PR to restore this as it was intended.

taleinat · 2020-09-19T15:50:08Z

See also another duplicate of this issue, bpo-40049.

JulienPalard · 2020-11-25T09:23:26Z

New changeset 4fedd71 by Julien Palard in branch 'master':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
4fedd71

miss-islington · 2020-11-25T09:53:00Z

New changeset 9d2c2a8 by Miss Islington (bot) in branch '3.9':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
9d2c2a8

miss-islington · 2020-11-25T10:01:13Z

New changeset bda2e68 by Miss Islington (bot) in branch '3.8':
bpo-12800: tarfile: Restore fix from 011525e (GH-21409)
bda2e68

adunand mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Aug 20, 2011

gustaebel mannequin self-assigned this Sep 12, 2011

websurfer5 mannequin added 3.7 (EOL) end of life 3.8 only security fixes labels May 9, 2019

taleinat added 3.9 only security fixes 3.10 only security fixes labels Sep 19, 2020

JulienPalard closed this as completed Dec 18, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009

'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009

adunand mannequin commented Aug 20, 2011

adunand mannequin commented Aug 20, 2011

BreamoreBoy mannequin commented Jun 25, 2014

serhiy-storchaka commented Jun 26, 2014

andrewgarner mannequin commented Jul 16, 2014

websurfer5 mannequin commented May 9, 2019

catlee mannequin commented May 16, 2020

JulienPalard commented May 17, 2020

catlee mannequin commented May 17, 2020

JulienPalard commented Jul 9, 2020

taleinat commented Sep 19, 2020

JulienPalard commented Nov 25, 2020

miss-islington commented Nov 25, 2020

miss-islington commented Nov 25, 2020

'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009

'tarfile.StreamError: seeking backwards is not allowed' when extract symlink #57009

Comments

adunand mannequin commented Aug 20, 2011

adunand mannequin commented Aug 20, 2011

BreamoreBoy mannequin commented Jun 25, 2014

serhiy-storchaka commented Jun 26, 2014

andrewgarner mannequin commented Jul 16, 2014

websurfer5 mannequin commented May 9, 2019

catlee mannequin commented May 16, 2020

JulienPalard commented May 17, 2020

catlee mannequin commented May 17, 2020

JulienPalard commented Jul 9, 2020

taleinat commented Sep 19, 2020

JulienPalard commented Nov 25, 2020

miss-islington commented Nov 25, 2020

miss-islington commented Nov 25, 2020