Author kulikjak
Recipients kulikjak
Date 2019-06-11.13:10:04
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1560258604.43.0.910483303566.issue37232@roundup.psfhosted.org>
In-reply-to
Content
When building and installing Python 3.8 on our sparc machine, the build breaks during the compileall stage with [Error 24] Too many open files. The problem is due to the recently enabled parallel compilation (issue36786).

When -j0 is passed to the compileall, executor starts as many threads as there are cpus, which is problem on our sparc machine with 512 CPUs and rather low ulimit on number of opened files. Compilation fails and is not able to recover from that. For the same reason, test_compileall also breaks and never ends.

I am able to fix both of these by patching both Makefile.pre.in and test_compileall.py with -j32 (instead of -j0), but that is not the best solution.

I think that the best solution to this would be to make compileall.py check for this limit and consider that when creating ProcessPoolExecutor (or, maybe even better, to make it recover when this problem occurs).
History
Date User Action Args
2019-06-11 13:10:04kulikjaksetrecipients: + kulikjak
2019-06-11 13:10:04kulikjaksetmessageid: <1560258604.43.0.910483303566.issue37232@roundup.psfhosted.org>
2019-06-11 13:10:04kulikjaklinkissue37232 messages
2019-06-11 13:10:04kulikjakcreate