This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_mmap uses cruel and unusual amounts of disk space
Type: resource usage Stage: needs patch
Components: Tests Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: larry, pitrou
Priority: normal Keywords:

Created on 2015-04-14 18:17 by larry, last changed 2022-04-11 14:58 by admin.

Messages (6)
msg240974 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-04-14 18:17
My laptop is running 64-bit Linux (14.10).  It has 4.6GB of free disk space.  Naturally that's not enough to run test_mmap.

When I run the test suite, test_mmap consumes all available disk space, then fails.  (Hopefully freeing all its temporary files!)  If I used "-j" to run more than one test at a time, this usually means *other* tests fail too, because I'm running multiple tests in parallel and there are plenty of other tests that require, y'know... any disk space whatsoever.

The documentation for the test suite ("./python -m test -h") says that "-u largefile" allows tests that use more than 2GB.  Surely test_mmap's delicious 800PB tests should be marked largefile-enabled-only?

I'd like to see this fix backported to 3.4 too.  And if 2.7 shows this behavior, maybe there too.
msg241044 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-04-14 22:14
What is your filesystem?
msg241045 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-04-14 22:19
By the way, I'm pretty sure test_mmap doesn't require 4GB of disk space here - Ubuntu 14.10 -, since it runs in only 0.071s; also it runs fine from a partition with only 3GB free space.
msg241056 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-04-14 23:20
ZFS.
msg241058 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-04-14 23:34
Perhaps ZFS doesn't support sparse files? Or perhaps there's another way to convince it to create a sparse file?

How long does test_mmap take to run on your machine?
msg241075 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-04-15 03:37
Wall time was 3 minutes 38 seconds.

% time ./python -m test test_mmap
[1/1] test_mmap
test test_mmap failed -- multiple errors occurred; run in verbose mode for details
1 test failed:
    test_mmap
./python -m test test_mmap  0.40s user 22.41s system 10% cpu 3:37.40 total
History
Date User Action Args
2022-04-11 14:58:15adminsetgithub: 68141
2015-04-15 03:37:31larrysetmessages: + msg241075
2015-04-14 23:34:38pitrousetmessages: + msg241058
2015-04-14 23:20:27larrysetmessages: + msg241056
2015-04-14 22:19:44pitrousetmessages: + msg241045
2015-04-14 22:14:29pitrousetnosy: + pitrou
messages: + msg241044
2015-04-14 18:17:35larrycreate