classification
Title: tracemalloc add a memory limit feature
Type: Stage:
Components: Versions: Python 3.5
process
Status: closed Resolution: out of date
Dependencies: 19835 Superseder:
Assigned To: Nosy List: Arfrever, neologix, python-dev, vstinner
Priority: normal Keywords: patch

Created on 2013-11-27 18:03 by vstinner, last changed 2015-10-02 21:08 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
tracemalloc_memory_limit.patch vstinner, 2013-11-27 18:03 review
tracemalloc_memory_limit-2.patch vstinner, 2013-12-04 01:00 review
test_limit.patch vstinner, 2013-12-04 01:04 review
Messages (5)
msg204605 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-11-27 18:03
It would be nice to add a memory_limit feature to the tracemalloc to make memory allocation fails if it would make the traced memory greater than the limit.

See also the pyfailmalloc project which is similar but different:
https://bitbucket.org/haypo/pyfailmalloc

pyfailmalloc doesn't count allocated bytes, it schedules an error in N allocations where N is a random number. And pyfailmalloc only schedules one error, whereas I propose to make all memory allocations fails until the limit is respected.

So if you only 0 bytes free (according to the limit), all memory allocations fail until some bytes are freed.

Python currently behaves badly under very low memory condition.

See also the issue #19437.
msg205155 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-03 21:15
This feature cannot be used without a reliable PyErr_NoMemory(): I add issue #19835 as a dependency.
msg205182 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-04 01:00
Updated patch.
msg205183 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-04 01:04
test_limit.patch: Patch to test the memory_limit on the Python test suite. tracemalloc_memory_limit-2.patch and unittest_leak.patch (of issue #19880) are required to test it.
msg205679 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-12-09 11:40
New changeset 45442f2a2494 by Victor Stinner in branch 'default':
Issue #19817: Fix print_exception(), clear the exception on error
http://hg.python.org/cpython/rev/45442f2a2494
History
Date User Action Args
2015-10-02 21:08:14vstinnersetstatus: open -> closed
resolution: out of date
2015-04-06 16:44:43Arfreversetnosy: + Arfrever
2013-12-09 11:40:36python-devsetnosy: + python-dev
messages: + msg205679
2013-12-04 01:04:46vstinnersetfiles: + test_limit.patch

messages: + msg205183
2013-12-04 01:01:00vstinnersetfiles: + tracemalloc_memory_limit-2.patch

messages: + msg205182
2013-12-03 21:15:00vstinnersetdependencies: + Add a MemoryError singleton to fix an unlimited loop when the memory is exhausted
messages: + msg205155
2013-11-27 18:03:12vstinnercreate