This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Taking sum of massive list comprehension results in total system crash.
Type: crash Stage: resolved
Components: Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: U9dB37BPZx, christian.heimes, mark.dickinson
Priority: normal Keywords:

Created on 2021-02-08 06:23 by U9dB37BPZx, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg386615 - (view) Author: George (U9dB37BPZx) Date: 2021-02-08 06:23
I tried running the following command in the interpreter, and without fail it will completely crash my entire computer.

Python 3.8.7 (default, Jan 20 2021, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> sum([i for i in range(10**8 + 10**9)])

My kernel is 5.10.12-100

I am running on Fedora 32 with Cinnamon Desktop if it helps. Let me know if you need any other information. 

I eagerly await resolution of this bug, as I want to know the result of the computation. Unless, of course it is harebrained, then we should ignore because Guido already discussed such computations.

George
msg386616 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-02-08 06:40
You are using a list comprehension that consumes a LOT of memory very fast. The line requires more physical RAM than available on a typical user system. This causes your computer to become unresponsive to input.

You can rewrite tie list comprehension as generator expression:

    sum(i for i in range(10**8 + 10**9))

It will consume far less memory, but it's slow and inefficient. You could use the triangular number algorithm instead:

    n = 10**8 + 10**9
    n * (n-1) // 2
msg386617 - (view) Author: George (U9dB37BPZx) Date: 2021-02-08 06:56
Thanks Christian for the solutions. I am guessing that since it is my entire machine that crashes, and python is not simply killed for requesting so much memory, that this is an operating system problem. Should I submit this as a bug to the kernel project then?

George
msg386638 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-02-08 18:11
The list you're attempting to create needs around 41GB of RAM (28 bytes for the actual object, rounded up to 32 bytes per object for alignment reasons, plus 8 bytes for each pointer in the list). Assuming you don't have that much memory on your system, it'll likely start swapping at that point.

> Should I submit this as a bug to the kernel project then?

That's up to you. The effect is well known and well documented - see for example https://bugzilla.redhat.com/show_bug.cgi?id=1577528 .

You may want to consider using `ulimit` to limit the amount of RAM that the Python process can use.

In any case, closing here, since this isn't a Python bug.
msg386640 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-02-08 18:12
> 28 bytes for the actual object

Gah. I should proof-read before hitting "Submit Changes". That should say "28 bytes for each int object".
History
Date User Action Args
2022-04-11 14:59:41adminsetgithub: 87327
2021-02-08 18:12:56mark.dickinsonsetmessages: + msg386640
2021-02-08 18:11:55mark.dickinsonsetstatus: open -> closed

nosy: + mark.dickinson
messages: + msg386638

resolution: not a bug
stage: resolved
2021-02-08 06:56:39U9dB37BPZxsetmessages: + msg386617
2021-02-08 06:40:03christian.heimessetnosy: + christian.heimes
messages: + msg386616
2021-02-08 06:23:08U9dB37BPZxcreate