classification
Title: Add support for custom compressor in tarfile
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Mariatta, insomniacslk, lars.gustaebel, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-07-24 21:05 by insomniacslk, last changed 2017-07-28 08:25 by insomniacslk.

Pull Requests
URL Status Linked Edit
PR 2734 open insomniacslk, 2017-07-24 21:05
Messages (5)
msg299017 - (view) Author: insomniacslk (insomniacslk) * Date: 2017-07-24 21:05
Tarfile would benefit from exposing custom compressors. At the moment the only way to use something that is not gzip/bzip/lzma is to separate the archiving and compression steps.

A possible approach is to pass a custom compression function to `tarfile.TarFile.open`. However the current interface is not clean enough to be exposed. I have made a very conservative change via a pull request on GitHub, see https://github.com/python/cpython/pull/2734 . Some additional considerations can be found there.

A further step could require a simplified interface that only involves file name, file-like object and compression level, and returns a file-like object to read the compressed data from. For example:

def my_compressor(name, fileobj=None, compresslevel=9):
    # compression happens here
    return filelike_object


This further step is not captured in the pull request, but I can iterate and update the diff.
msg299356 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2017-07-27 23:31
Adding lars.gustaebel who is listed as maintainer for tarfile.
msg299357 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2017-07-27 23:35
Adding serhiy.storchaka who recently made changes to tarfile.
msg299360 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-07-28 04:26
You can pass a custom file object that implements a custom compression to the TarFile constructor. I don't see a need in other way.
msg299375 - (view) Author: insomniacslk (insomniacslk) * Date: 2017-07-28 08:25
The documentation should be updated to suggest that a custom file-like object can be passed to have custom compressors then.

However this sounds very hacky and definitely not a feature that can be easily passed to the user. Hence the reason for a cleaner, explicitly advertised interface.
History
Date User Action Args
2017-07-28 08:25:38insomniacslksetmessages: + msg299375
2017-07-28 04:26:44serhiy.storchakasetmessages: + msg299360
2017-07-28 00:25:43Mariattasetversions: - Python 3.6
2017-07-27 23:35:17Mariattasetnosy: + serhiy.storchaka
messages: + msg299357
2017-07-27 23:31:57Mariattasetstage: patch review
2017-07-27 23:31:48Mariattasetnosy: + Mariatta, lars.gustaebel
messages: + msg299356
2017-07-24 21:05:21insomniacslkcreate