Title: tarfile doesn't handle sysfs well
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.3, Python 3.4, Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: lars.gustaebel Nosy List: Yoni.Tsafir, guyrozendorn, lars.gustaebel
Priority: normal Keywords:

Created on 2010-12-22 17:05 by Yoni.Tsafir, last changed 2013-07-06 00:34 by christian.heimes.

Messages (2)
msg124514 - (view) Author: Yoni Tsafir (Yoni.Tsafir) Date: 2010-12-22 17:05
When I try to add a special file from sys, e.g.:
/sys/class/scsi_host/host0/cmd_per_lun (which is reported of size 4096 but actually reading it will return only several bytes of a result), I get the following exception:

Traceback (most recent call last):
  File "/opt/xpyv/lib/", line 1975, in add
    self.addfile(tarinfo, f)
  File "/opt/xpyv/lib/", line 2004, in addfile
    copyfileobj(fileobj, self.fileobj, tarinfo.size)
  File "/opt/xpyv/lib/", line 287, in copyfileobj
    raise IOError("end of file reached")
IOError: end of file reached

Notice what happens if I try to add the file with regular tar:
root@buzaglo # tar cvzf /tmp/blat.tgz /sys/class/scsi_host/host0/cmd_per_lun
tar: Removing leading `/' from member names
tar: /sys/class/scsi_host/host0/cmd_per_lun: File shrank by 4094 bytes; padding with zeros
tar: Error exit delayed from previous errors

So it handles the issue by padding the rest of the file size with zeros.

I think this should be the behavior as well, instead of throwing an IOError.
msg171090 - (view) Author: Guy Rozendorn (guyrozendorn) Date: 2012-09-24 01:53
Here's a test case that re-creates this issue.
I chose to use mocks instead of sample files from sysfs so it would be simpler to run, it can be easily changed to use a file from sysfs.

The following code runs on Python2.7, requires the mock library

from unittest import TestCase
from tempfile import mkstemp
from mock import patch, Mock
from os import close, remove, write, stat
from posix import stat_result
from tarfile import TarFile

def fake_st_size_side_effect(*args, **kwargs):
    src, = args
    stats = stat(src)
    return stat_result((stats.st_mode, stats.st_ino, stats.st_dev, stats.st_nlink,
                       stats.st_uid, stats.st_gid, stats.st_size + 10,
                       stats.st_atime, stats.st_mtime, stats.st_ctime))

class Issue10760TestCase(TestCase):
    def setUp(self):
        fd, self.src = mkstemp()
        write(fd, '\x00' * 4)
        fd, self.dst = mkstemp()

    def test(self):
        with patch("os.lstat") as lstat:
            lstat.side_effect = fake_st_size_side_effect
            tar_file =, 'w:gz')

Date User Action Args
2013-07-06 00:34:55christian.heimessetversions: + Python 3.3, Python 3.4
2012-09-24 01:53:53guyrozendornsetmessages: + msg171090
2011-07-22 19:45:05terry.reedysetstage: test needed
versions: + Python 2.7, - Python 2.6
2011-06-26 22:50:19guyrozendornsetnosy: + guyrozendorn
2011-01-04 08:25:35lars.gustaebelsetassignee: lars.gustaebel

components: + Library (Lib), - None
nosy: + lars.gustaebel
2010-12-22 17:05:51Yoni.Tsafircreate