Author Ben Cipollini
Recipients Ben Cipollini
Date 2015-11-14.19:48:21
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1447530501.75.0.968129152166.issue25626@psf.upfronthosting.co.za>
In-reply-to
Content
Gzip fails when opening a file more than 2**32 bytes. This is a new issue in Python 3.5.

We hit this opening large neuroimaging files from the Human Connectome Project. See https://github.com/nipy/nibabel/issues/362 for more details.

When size is > 2**32, we get the following error on Python 3.5:

/usr/lib/python3.5/gzip.py in read(self, size)
    467             buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
    468 
--> 469             uncompress = self._decompressor.decompress(buf, size)
    470             if self._decompressor.unconsumed_tail != b"":
    471                 self._fp.prepend(self._decompressor.unconsumed_tail)

OverflowError: Python int too large for C unsigned int
History
Date User Action Args
2015-11-14 19:48:21Ben Cipollinisetrecipients: + Ben Cipollini
2015-11-14 19:48:21Ben Cipollinisetmessageid: <1447530501.75.0.968129152166.issue25626@psf.upfronthosting.co.za>
2015-11-14 19:48:21Ben Cipollinilinkissue25626 messages
2015-11-14 19:48:21Ben Cipollinicreate