classification
Title: tarfile doesn't handle sysfs well
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: lars.gustaebel Nosy List: Yoni.Tsafir, guyrozendorn, lars.gustaebel
Priority: normal Keywords:

Created on 2010-12-22 17:05 by Yoni.Tsafir, last changed 2013-07-06 00:34 by christian.heimes.

Messages (2)
msg124514 - (view) Author: Yoni Tsafir (Yoni.Tsafir) Date: 2010-12-22 17:05
When I try to add a special file from sys, e.g.:
/sys/class/scsi_host/host0/cmd_per_lun (which is reported of size 4096 but actually reading it will return only several bytes of a result), I get the following exception:

Traceback (most recent call last):
  File "/opt/xpyv/lib/python26.zip/tarfile.py", line 1975, in add
    self.addfile(tarinfo, f)
  File "/opt/xpyv/lib/python26.zip/tarfile.py", line 2004, in addfile
    copyfileobj(fileobj, self.fileobj, tarinfo.size)
  File "/opt/xpyv/lib/python26.zip/tarfile.py", line 287, in copyfileobj
    raise IOError("end of file reached")
IOError: end of file reached

Notice what happens if I try to add the file with regular tar:
root@buzaglo # tar cvzf /tmp/blat.tgz /sys/class/scsi_host/host0/cmd_per_lun
tar: Removing leading `/' from member names
/sys/class/scsi_host/host0/cmd_per_lun
tar: /sys/class/scsi_host/host0/cmd_per_lun: File shrank by 4094 bytes; padding with zeros
tar: Error exit delayed from previous errors

So it handles the issue by padding the rest of the file size with zeros.

I think this should be the behavior as well, instead of throwing an IOError.
msg171090 - (view) Author: Guy Rozendorn (guyrozendorn) Date: 2012-09-24 01:53
Here's a test case that re-creates this issue.
I chose to use mocks instead of sample files from sysfs so it would be simpler to run, it can be easily changed to use a file from sysfs.

The following code runs on Python2.7, requires the mock library

{code}
from unittest import TestCase
from tempfile import mkstemp
from mock import patch, Mock
from os import close, remove, write, stat
from posix import stat_result
from tarfile import TarFile

def fake_st_size_side_effect(*args, **kwargs):
    src, = args
    stats = stat(src)
    return stat_result((stats.st_mode, stats.st_ino, stats.st_dev, stats.st_nlink,
                       stats.st_uid, stats.st_gid, stats.st_size + 10,
                       stats.st_atime, stats.st_mtime, stats.st_ctime))

class Issue10760TestCase(TestCase):
    def setUp(self):
        fd, self.src = mkstemp()
        write(fd, '\x00' * 4)
        close(fd)
        fd, self.dst = mkstemp()
        close(fd)

    def test(self):
        with patch("os.lstat") as lstat:
            lstat.side_effect = fake_st_size_side_effect
            tar_file = TarFile.open(self.dst, 'w:gz')
            tar_file.add(self.src)

{code}
History
Date User Action Args
2013-07-06 00:34:55christian.heimessetversions: + Python 3.3, Python 3.4
2012-09-24 01:53:53guyrozendornsetmessages: + msg171090
2011-07-22 19:45:05terry.reedysetstage: test needed
versions: + Python 2.7, - Python 2.6
2011-06-26 22:50:19guyrozendornsetnosy: + guyrozendorn
2011-01-04 08:25:35lars.gustaebelsetassignee: lars.gustaebel

components: + Library (Lib), - None
nosy: + lars.gustaebel
2010-12-22 17:05:51Yoni.Tsafircreate