Author neologix
Recipients josh.r, jtaylor, neologix, njs, pitrou, skrah, vstinner
Date 2014-04-27.18:31:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAH_1eM2CSsyTm=SfLT5H47HNxz0hMskur7D0_-ykiRSEcJZvCw@mail.gmail.com>
In-reply-to <CAH_1eM3jyWjdVrF=LuEkN987p-KLERpxKjUW1j4HNVsakUoibg@mail.gmail.com>
Content
Alright, it bothered me so I wrote a small C testcase (attached),
which calls malloc in a loop, and can call memset upon the allocated
block right after allocation:

$ gcc -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 3050MB
$ gcc -DDO_MEMSET -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 2130MB

Without memset, the kernel happily allocates until we reach the 3GB
user address space limit.
With memset, it bails out way before.

I don't know what this'll give on 64-bit, but I assume one should get
comparable result.

I would guess that the reason why the Python list allocation fails is
because of the exponential allocation scheme: since memory is
allocated in large chunks before being used, the kernel happily
overallocates.
With a more progressive allocation+usage, it should return ENOMEM at some point.

Anyway, that's probably off-topic!
Files
File name Uploaded
test.c neologix, 2014-04-27.18:31:49
History
Date User Action Args
2014-04-27 18:31:50neologixsetrecipients: + neologix, pitrou, vstinner, njs, skrah, jtaylor, josh.r
2014-04-27 18:31:50neologixlinkissue21233 messages
2014-04-27 18:31:49neologixcreate