This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author marko-tuononen
Recipients marko-tuononen
Date 2021-08-12.11:43:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1628768592.88.0.0306570478346.issue44899@roundup.psfhosted.org>
In-reply-to
Content
I have a use case where I need to create a tar archive from a collection of potentially changing files. I need to use system resources sparingly and because of that it is not possible to first make a copy of the files.

Current state of the tarfile library: Creating a tar archive is interrupted with an OSError "unexpected end of data" (example below), if any of the files changes when it is collected. Using the tarfile library in streaming mode does not work either. You might find this bug report relevant: https://bugs.python.org/issue26877

   File "/usr/lib64/python3.7/tarfile.py", line 1946, in add
     self.addfile(tarinfo, f)
   File "/usr/lib64/python3.7/tarfile.py", line 1974, in addfile
     copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
   File "/usr/lib64/python3.7/tarfile.py", line 249, in copyfileobj
     raise exception("unexpected end of data")
   OSError: unexpected end of data

Target state of the tarfile library: Creating a tar archive is not interrupted even if a file changes while collected. The tarfile library's add() method would just return an exit value indicating that some files were changed while being archived. See e.g. how GNU tar handles similar situation: https://man7.org/linux/man-pages/man1/tar.1.html#RETURN_VALUE
History
Date User Action Args
2021-08-12 11:43:12marko-tuononensetrecipients: + marko-tuononen
2021-08-12 11:43:12marko-tuononensetmessageid: <1628768592.88.0.0306570478346.issue44899@roundup.psfhosted.org>
2021-08-12 11:43:12marko-tuononenlinkissue44899 messages
2021-08-12 11:43:12marko-tuononencreate