classification
Title: zlibmodule.c isn't 64-bit clean
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.1, Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gregory.p.smith, haypo, loewis, nadeem.vawda, pitrou, python-dev, rcoyner
Priority: normal Keywords:

Created on 2010-05-07 17:20 by pitrou, last changed 2011-05-14 23:09 by nadeem.vawda. This issue is now closed.

Messages (8)
msg105216 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-07 17:20
zlibmodule.c doesn't define PY_SSIZE_T_CLEAN, and uses lots of "int" variables for length values internally.
Besides, the zlib itself uses "uInt" and "uLong" variables in its z_stream structure (rather than something guaranteed to be at least the width of a pointer).

On large input data, this will conspire to produce bogus results or crashes. For example:

>>> s = 'x' * (4 * 1024**3 + 100)
>>> t = zlib.compress(s, 1)
>>> len(t)
12
>>> len(zlib.decompress(t))
100
msg106584 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-05-26 21:56
See also a related issue: #8651.
msg125283 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-01-04 02:08
Fixed by r87729.

Wait for the buildbots before backporting to other versions.
msg128984 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-02-21 19:28
Woops, I removed a duplicate message of Antoine. in this issue instead of #8651. Removed message:
-----
> Fixed by r87729.

This only addresses the compress() and decompress() functions, but e.g. crc32() and adler32() are also touched by this issue.
-----
msg131611 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-03-21 03:05
> This only addresses the compress() and decompress() functions, 
> but e.g. crc32() and adler32() are also touched by this issue.

crc32() and adler32() were fixed by #10276.

PyZlib_objcompress() and PyZlib_objdecompress() should be fixed.
msg135988 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-14 20:34
New changeset 850624632e9a by Nadeem Vawda in branch '3.1':
Issue #8650: Backport 64-bit safety fixes for compress() and decompress().
http://hg.python.org/cpython/rev/850624632e9a

New changeset 99a3ce22c1ff by Nadeem Vawda in branch '3.2':
Null merge (#8650): zlib.compress() and .decompress() are already 64-bit safe.
http://hg.python.org/cpython/rev/99a3ce22c1ff

New changeset dea5fcdc9ecd by Nadeem Vawda in branch 'default':
Null merge (#8650): zlib.compress() and .decompress() are already 64-bit safe.
http://hg.python.org/cpython/rev/dea5fcdc9ecd
msg135998 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-14 22:24
New changeset 0b5211d19a14 by Nadeem Vawda in branch '3.1':
Issue #8650: Make zlib.[de]compressobj().[de]compress() 64-bit clean.
http://hg.python.org/cpython/rev/0b5211d19a14

New changeset 3ff862d05d18 by Nadeem Vawda in branch '3.2':
Merge: #8650: Make zlib.[de]compressobj().[de]compress() 64-bit clean.
http://hg.python.org/cpython/rev/3ff862d05d18

New changeset f86a22b6ab58 by Nadeem Vawda in branch 'default':
Merge: #8650: Make zlib.[de]compressobj().[de]compress() 64-bit clean.
http://hg.python.org/cpython/rev/f86a22b6ab58
msg136002 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2011-05-14 23:09
Fixed in 3.1, 3.2 and 3.3.

2.7 no longer has this problem -- the fix for issue8651 ensures that the
functions cannot receive inputs large enough to be problematic (since we
don't define PY_SSIZE_T_CLEAN).
History
Date User Action Args
2011-05-14 23:09:19nadeem.vawdasetstatus: open -> closed
versions: + Python 3.1, Python 3.2, Python 3.3, - Python 2.7
messages: + msg136002

resolution: fixed
stage: needs patch -> resolved
2011-05-14 22:24:55python-devsetmessages: + msg135998
2011-05-14 20:34:49python-devsetnosy: + python-dev
messages: + msg135988
2011-03-21 03:05:08hayposetnosy: loewis, gregory.p.smith, pitrou, haypo, nadeem.vawda, rcoyner
messages: + msg131611
2011-02-21 19:28:23hayposetnosy: loewis, gregory.p.smith, pitrou, haypo, nadeem.vawda, rcoyner
messages: + msg128984
2011-02-21 19:26:19hayposetnosy: loewis, gregory.p.smith, pitrou, haypo, nadeem.vawda, rcoyner
messages: - msg128975
2011-02-21 19:19:11nadeem.vawdasetnosy: + nadeem.vawda
2011-02-21 18:26:30pitrousetnosy: loewis, gregory.p.smith, pitrou, haypo, rcoyner
messages: + msg128975
versions: - Python 2.6, Python 3.1, Python 3.2
2011-01-04 02:08:58hayposetnosy: loewis, gregory.p.smith, pitrou, haypo, rcoyner
messages: + msg125283
2010-08-15 18:13:03loewissetmessages: - msg113981
2010-08-15 18:00:27loewissetnosy: + loewis
messages: + msg113981
2010-05-29 15:52:09rcoynersetnosy: + rcoyner
2010-05-26 21:56:30hayposetmessages: + msg106584
2010-05-07 17:24:38pitrousetnosy: + haypo
2010-05-07 17:20:59pitroucreate