This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Potential division by zero and integer overflow in allocator wrappers
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, izbyshev, miss-islington, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-10-28 12:13 by izbyshev, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 10174 merged izbyshev, 2018-10-28 12:16
PR 10198 merged miss-islington, 2018-10-28 20:28
PR 10199 merged miss-islington, 2018-10-28 20:29
Messages (14)
msg328686 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-10-28 12:13
BZ2_Malloc() checks for size < 0 at https://github.com/python/cpython/blob/6015cc50bc38b9e920ce4986ee10658eaa14f561/Modules/_bz2module.c#L278 , but doesn't check for size == 0 before dividing by it:

    if (items < 0 || size < 0)
        return NULL;
    if ((size_t)items > (size_t)PY_SSIZE_T_MAX / (size_t)size)
        return NULL;

Reported by Svace static analyzer.
msg328693 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 15:31
Check other wrappers to memory allocators:

* zlib: "zst.zalloc = PyZlib_Malloc" which calls PyMem_RawMalloc
* _lzma: "self->alloc.alloc = PyLzma_Malloc" which calls PyMem_RawMalloc
* _bz2: "bzalloc = BZ2_Malloc" which calls PyMem_RawMalloc()

https://bugs.python.org/issue35056#msg328533
msg328697 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-10-28 16:04
May be we should add a new function (_PyMem_RawMallocItems?) that does the same checks as PyMem_RawCalloc, but doesn't zero-initialize memory?
msg328698 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 16:07
> May be we should add a new function (_PyMem_RawMallocItems?) that does the same checks as PyMem_RawCalloc, but doesn't zero-initialize memory?

Please don't add new functions to the Python memory allocators. We already have too many of them :-(
https://docs.python.org/dev/c-api/memory.html

PyZlib_Malloc, PyLzma_Malloc and BZ2_Malloc exists because they use different types: 2 unsigned int (zlib), 2 size_t (lzma), 2 int (bz2). PyMem_RawMalloc() expects a single size_t.

IMHO it's fine to have a function of 5 lines of code in each module, since each module uses a different C type.
msg328704 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 16:45
New changeset 3d4fabb2a424cb04ae446ebe4428090c386f45a5 by Victor Stinner (Alexey Izbyshev) in branch 'master':
bpo-35090: Fix potential division by zero in allocator wrappers (GH-10174)
https://github.com/python/cpython/commit/3d4fabb2a424cb04ae446ebe4428090c386f45a5
msg328705 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 16:47
I don't want to backport this change. IMHO these wrappers are never called with 0. Otherwise, I'm sure that someone would report the crash...

But I applied the change anyway, just to avoid future reports of static analyzers :-) And the cast doesn't hurt :-)
msg328706 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-10-28 16:51
Thanks, Victor. Regarding backporting, what about integer overflow? Do you think it's guaranteed that the multiple of items and size always fits in 32-bit types, in case of BZ2_Malloc and PyZlib_Malloc?
msg328708 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-10-28 16:53
Sorry for changing the status, it's browser caching again.
msg328709 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 16:57
> Regarding backporting, what about integer overflow? Do you think it's guaranteed that the multiple of items and size always fits in 32-bit types, in case of BZ2_Malloc and PyZlib_Malloc?

I don't know. Serhiy: What do you think?
msg328731 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-10-28 19:43
I think it is worth to backport. Perhaps it is possible even to create a reproducer for integer overflow, but I afraid it will be too slow for using in tests.
msg328740 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 20:31
Serhiy: "I think it is worth to backport."

Ok. I asked the bot to create 3.6 and 3.7 backports, and I just approved them.

I checked: Python 2.7 is not affected.
msg328747 - (view) Author: miss-islington (miss-islington) Date: 2018-10-28 20:47
New changeset 1d7d165e3c3f07518e6b5bfb57f1fd460cd4bbf2 by Miss Islington (bot) in branch '3.7':
bpo-35090: Fix potential division by zero in allocator wrappers (GH-10174)
https://github.com/python/cpython/commit/1d7d165e3c3f07518e6b5bfb57f1fd460cd4bbf2
msg328750 - (view) Author: miss-islington (miss-islington) Date: 2018-10-28 20:59
New changeset fd0a3bce6e917b5853c809a309c1513acc176f56 by Miss Islington (bot) in branch '3.6':
bpo-35090: Fix potential division by zero in allocator wrappers (GH-10174)
https://github.com/python/cpython/commit/fd0a3bce6e917b5853c809a309c1513acc176f56
msg328751 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-28 21:00
Ok, now we should be good :-) I close again the issue.

Note: Serhiy Stochaka considers that no NEWS entry is needed and I concur with him.
History
Date User Action Args
2022-04-11 14:59:07adminsetgithub: 79271
2018-10-28 21:00:22vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg328751

stage: patch review -> resolved
2018-10-28 20:59:17miss-islingtonsetmessages: + msg328750
2018-10-28 20:47:02miss-islingtonsetnosy: + miss-islington
messages: + msg328747
2018-10-28 20:31:15vstinnersetmessages: + msg328740
versions: + Python 3.6, Python 3.7
2018-10-28 20:29:03miss-islingtonsetpull_requests: + pull_request9515
2018-10-28 20:28:55miss-islingtonsetstage: resolved -> patch review
pull_requests: + pull_request9514
2018-10-28 20:28:39vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
2018-10-28 19:43:39serhiy.storchakasetmessages: + msg328731
2018-10-28 16:57:56vstinnersetmessages: + msg328709
2018-10-28 16:53:04izbyshevsetstatus: open -> closed

resolution: fixed
messages: + msg328708
versions: - Python 3.6, Python 3.7
2018-10-28 16:51:23izbyshevsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg328706

versions: + Python 3.6, Python 3.7
2018-10-28 16:47:05vstinnersetstatus: open -> closed
versions: - Python 3.6, Python 3.7
messages: + msg328705

resolution: fixed
stage: patch review -> resolved
2018-10-28 16:45:54vstinnersetmessages: + msg328704
2018-10-28 16:24:23izbyshevsettitle: bz2: Potential division by zero in BZ2_Malloc() -> Potential division by zero and integer overflow in allocator wrappers
2018-10-28 16:07:47vstinnersetmessages: + msg328698
2018-10-28 16:04:59izbyshevsetmessages: + msg328697
2018-10-28 15:31:18vstinnersetmessages: + msg328693
2018-10-28 12:16:57izbyshevsetkeywords: + patch
stage: patch review
pull_requests: + pull_request9497
2018-10-28 12:13:54izbyshevcreate