classification
Title: test_list.test_overflow crashes Win64
Type: resource usage Stage: resolved
Components: Tests Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: BreamoreBoy, anselm.kruis, pitrou, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-05-17 23:46 by anselm.kruis, last changed 2014-08-10 12:23 by serhiy.storchaka. This issue is now closed.

Messages (9)
msg189491 - (view) Author: Anselm Kruis (anselm.kruis) * Date: 2013-05-17 23:46
I installed Python 2.7.5 including tests using the MSI installer from http://www.python.org/ftp/python/2.7.5/python-2.7.5.amd64.msi Running 

python.exe -m test.regrtest -v test_list

consumes all available memory and renders the my system completely unusable. I had to reboot. The problem is caused by the test case "test_overflow". If you run the test with an imposed memory limit, e.g. by using ulimitnt (http://code.google.com/p/ulimitnt/) , test_overflow passes.

I propose to skip this test on Win64.
msg189509 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-18 12:24
I take it you have more than 16GB of RAM?
What happens if you replace "sys.maxint" with "sys.maxsize" in test_overflow?
msg189510 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-05-18 12:25
Note: "test_overflow" can be found in Python2.7/Lib/test/test_list.py
msg189728 - (view) Author: Anselm Kruis (anselm.kruis) * Date: 2013-05-21 08:28
> I take it you have more than 16GB of RAM?
I used a system with 16GB Ram.

> What happens if you replace "sys.maxint" with "sys.maxsize" in test_overflow?
The test passes. Both mul and imul raise MemoryError.
msg224441 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-31 19:29
What if anything needs to be done here?
msg224517 - (view) Author: Anselm Kruis (anselm.kruis) * Date: 2014-08-01 20:21
Currently, if you run the test suite of a 64bit Windows python installation it consumes all your memory and - depending on your RAM and swap space - you system becomes unusable. That's a behaviour nobody expects.

I didn't look into the implementation of the test case test_list. Therefore I can't make a proposal what to change. Obviously there are at least two options: 
- skip the test
- replace "sys.maxint" with "sys.maxsize" in test_overflow
msg224520 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-01 21:06
Definitely this test should be decorated as bigmemtest. Here are patches.
msg224533 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-02 04:51
I was wrong. Here should be just sys.maxsize. With sys.maxint the test should consume about 32 GB of memory and then fail on assertion (imul requires even 40 GB, but the test fails earlier).
msg224940 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-08-06 15:32
New changeset 217485e80a32 by Serhiy Storchaka in branch '2.7':
Issue #18004: test_overflow in test_list by mistake consumed 40 GiB of memory
http://hg.python.org/cpython/rev/217485e80a32
History
Date User Action Args
2014-08-10 12:23:23serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2014-08-06 15:39:17serhiy.storchakasetassignee: serhiy.storchaka
stage: needs patch -> commit review
2014-08-06 15:32:19python-devsetnosy: + python-dev
messages: + msg224940
2014-08-02 04:51:43serhiy.storchakasetstage: patch review -> needs patch
messages: + msg224533
versions: - Python 3.4, Python 3.5
2014-08-02 04:46:15serhiy.storchakasetfiles: - test_list_overflow-2.7.patch
2014-08-02 04:45:31serhiy.storchakasetfiles: - test_list_overflow-3.4.patch
2014-08-01 21:08:31serhiy.storchakasetfiles: + test_list_overflow-2.7.patch
2014-08-01 21:06:42serhiy.storchakasetfiles: + test_list_overflow-3.4.patch

versions: + Python 3.4, Python 3.5
keywords: + patch
nosy: + serhiy.storchaka

messages: + msg224520
stage: patch review
2014-08-01 20:21:19anselm.kruissetmessages: + msg224517
2014-07-31 19:29:57BreamoreBoysetnosy: + BreamoreBoy
messages: + msg224441
2013-05-21 08:28:08anselm.kruissetmessages: + msg189728
2013-05-18 13:28:46arigosetnosy: - arigo
2013-05-18 12:25:48pitrousetmessages: + msg189510
2013-05-18 12:24:45pitrousetnosy: + pitrou, arigo
messages: + msg189509
2013-05-17 23:46:49anselm.kruiscreate