This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author martin.panter
Recipients Arfrever, martin.panter, nikratio, pitrou, serhiy.storchaka
Date 2015-03-09.00:51:16
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
I have decided not to reuse the same _DecompressReader for the "gzip" module. This is because the code for parsing the "gzip" header pulls in data by calling read(), and I would have to adapt this code to accept whatever data is pushed to it via a decompress() method. However I have rewritten the “gzip” module to use BufferedReader, removing its own readline() etc implementations.

I have split the BaseStream and DecompressReader classes into a new file called Lib/ Let me know if there is anything else to be done to add a new file in the Lib/ directory, or if you have any better ideas where these classes should live.

Other changes in the new patch (LZMAFile-etc.v4.patch):
* Fix documented GzipFile.peek(n) signature to match implementation
* Removed unused and inconsistent code paths in gzip._PaddedFile
* New code assumes zlib.decompressobj().flush() returns a limited amount of data, since it has no “max_length” parameter. In reality I think flush() does not do anything at all and should be deprecated or removed; see Issue 23200.

I hope that the patch does not need any more major changes, so it should be ready for review. There are a few more suggested enhancements that I have not implemented. While they would be nice, I would prefer to handle them in separate bugs and patches. I believe strengthening Python against gzip bombs is important, and any extra refactoring of the code will probably make things harder to review, more likely to break, and less likely to make it into 3.5.

Some possible enhancements I did not do:
* Support open(buffering=...) parameter, passing through to the [...]File(buffer_size=...) parameter
* open(buffering=0) returning an unbuffered reader object, probably just a direct DecompressReader instance
* detach() method on the BufferedIOBase file classes
* Further factoring out of a common CompressedFile base class
* Apply the buffer size parameter to write mode
* Rewrite the gzip module to use the common DecompressReader base class
Date User Action Args
2015-03-09 00:51:18martin.pantersetrecipients: + martin.panter, pitrou, Arfrever, nikratio, serhiy.storchaka
2015-03-09 00:51:17martin.pantersetmessageid: <>
2015-03-09 00:51:17martin.panterlinkissue23529 messages
2015-03-09 00:51:17martin.pantercreate