➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-09-23 07:48 by zmk, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 32000 | merged | gregory.p.smith, 2022-03-19 23:06 | |
| PR 32013 | merged | gregory.p.smith, 2022-03-20 20:31 | |
| PR 32015 | merged | gregory.p.smith, 2022-03-20 21:52 | |
| Messages (10) | |||
|---|---|---|---|
| msg352992 - (view) | Author: marko kreen (zmk) | Date: 2019-09-23 07:48 | |
Following code: ---------------------------- import binascii, zlib bigdata=memoryview(bytearray((1<<32) + 100)) print(binascii.crc32(bigdata)) crc = binascii.crc32(bigdata[:1000]) crc = binascii.crc32(bigdata[1000:], crc) print(crc) print(zlib.crc32(bigdata)) crc = zlib.crc32(bigdata[:1000]) crc = zlib.crc32(bigdata[1000:], crc) print(crc) --------------------------- Outputs with python3.7 $ python3.7 x.py 2575877834 2838121701 2838121701 2838121701 |
|||
| msg352993 - (view) | Author: Александр Семенов (iamsav) | Date: 2019-09-23 08:01 | |
I've got Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] 2838121701 2838121701 2838121701 2838121701 |
|||
| msg352994 - (view) | Author: marko kreen (zmk) | Date: 2019-09-23 08:16 | |
Looking at the code, the bug is dependant on USE_ZLIB_CRC32 define and it's unfixed in master. Cause is zlib crc32 that takes length as integer, zlibmodule has workaround, binascii has not. |
|||
| msg352995 - (view) | Author: marko kreen (zmk) | Date: 2019-09-23 08:49 | |
Found zlibmodule fix: https://bugs.python.org/issue10276 |
|||
| msg415577 - (view) | Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2022-03-19 22:25 | |
``` $ python3.8 Python 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import binascii, zlib >>> bigdata=memoryview(bytearray((1<<32) + 100)) >>> >>> print(binascii.crc32(bigdata)) 2838121701 >>> crc = binascii.crc32(bigdata[:1000]) >>> crc = binascii.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> >>> print(zlib.crc32(bigdata)) 2838121701 >>> crc = zlib.crc32(bigdata[:1000]) >>> crc = zlib.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 ``` |
|||
| msg415578 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2022-03-19 22:35 | |
it depends on the build. USE_ZLIB_CRC32 causes it due to zlib's 32-bitness as noted my marko. $ ./python Python 3.11.0a6+ (heads/main-dirty:b3f2d4c8ba, Mar 19 2022, 15:32:04) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import binascii, zlib >>> bigdata=memoryview(bytearray((1<<32) + 100)) >>> >>> print(binascii.crc32(bigdata)) 2575877834 >>> crc = binascii.crc32(bigdata[:1000]) >>> crc = binascii.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> >>> print(zlib.crc32(bigdata)) 2838121701 >>> crc = zlib.crc32(bigdata[:1000]) >>> crc = zlib.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> |
|||
| msg415628 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2022-03-20 19:28 | |
New changeset 9d1c4d69dbc800ac344565119337fcf490cdc800 by Gregory P. Smith in branch 'main': bpo-38256: Fix binascii.crc32() when inputs are 4+GiB (GH-32000) https://github.com/python/cpython/commit/9d1c4d69dbc800ac344565119337fcf490cdc800 |
|||
| msg415636 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2022-03-20 21:47 | |
New changeset 4c989e19c84ec224655bbbde9422e16d4a838a80 by Gregory P. Smith in branch '3.10': [3.10] bpo-38256: Fix binascii.crc32 large input. (GH-32000) (GH-32013) https://github.com/python/cpython/commit/4c989e19c84ec224655bbbde9422e16d4a838a80 |
|||
| msg415667 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2022-03-21 06:34 | |
New changeset 58a7e130375776b192a99b013bc563205a639edc by Gregory P. Smith in branch '3.9': bpo-38256: Fix binascii.crc32 large input. (GH-32000) (GH-32013) (GH-32015) https://github.com/python/cpython/commit/58a7e130375776b192a99b013bc563205a639edc |
|||
| msg415668 - (view) | Author: Gregory P. Smith (gregory.p.smith) * |
Date: 2022-03-21 06:37 | |
3.8 and older are in security fix only mode. 3.9+ have been fixed. realistically this was a rare issue with multiple workarounds. The 3.9 and 3.10 fixes were strictly bugfix only. The 3.11 fix included some performance improvements. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:20 | admin | set | github: 82437 |
| 2022-03-21 06:37:04 | gregory.p.smith | set | status: open -> closed resolution: fixed messages: + msg415668 stage: patch review -> commit review |
| 2022-03-21 06:34:49 | gregory.p.smith | set | messages: + msg415667 |
| 2022-03-20 21:52:15 | gregory.p.smith | set | pull_requests: + pull_request30103 |
| 2022-03-20 21:47:05 | gregory.p.smith | set | messages: + msg415636 |
| 2022-03-20 20:31:12 | gregory.p.smith | set | pull_requests: + pull_request30101 |
| 2022-03-20 19:28:39 | gregory.p.smith | set | messages: + msg415628 |
| 2022-03-19 23:48:10 | gregory.p.smith | set | assignee: gregory.p.smith |
| 2022-03-19 23:06:19 | gregory.p.smith | set | keywords:
+ patch stage: needs patch -> patch review pull_requests: + pull_request30089 |
| 2022-03-19 22:35:31 | gregory.p.smith | set | status: closed -> open versions: + Python 3.8, Python 3.9, Python 3.10, Python 3.11 title: binascii.crc32 is not 64-bit clean -> binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32 messages: + msg415578 resolution: works for me -> (no value) stage: resolved -> needs patch |
| 2022-03-19 22:25:10 | gregory.p.smith | set | status: open -> closed versions: - Python 3.8, Python 3.9 nosy: + gregory.p.smith messages: + msg415577 resolution: works for me stage: resolved |
| 2019-09-23 09:18:28 | serhiy.storchaka | set | nosy:
+ nadeem.vawda |
| 2019-09-23 08:49:04 | zmk | set | messages:
+ msg352995 versions: + Python 3.8, Python 3.9 |
| 2019-09-23 08:16:36 | zmk | set | messages: + msg352994 |
| 2019-09-23 08:01:35 | iamsav | set | nosy:
+ iamsav messages: + msg352993 |
| 2019-09-23 07:48:21 | zmk | create | |