This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients dstufft, eric.araujo, vstinner
Date 2017-04-28.12:17:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1493381870.33.0.670258208626.issue30198@psf.upfronthosting.co.za>
In-reply-to
Content
Since Python 3.5, distutils is able to build extensions in parallel, nice enhancement! But setup.py of CPython is 2x slower in parallel mode when all modules are already built: (1 sec vs 500 ms).

Building extensions calls newer_group() which calls os.stat() 6,856 times. I wrote a cache for os.stat() but it has no impact on performance.

It seems like threads are fighting to death for the GIL in the os.stat() race...

Attached pull request calls newer_group() before spawning threads in parallel mode, so "setup.py build" takes the same time with and without parallel module, when all extensions are already built.

I didn't measure performance when all extensions must be built.
History
Date User Action Args
2017-04-28 12:17:50vstinnersetrecipients: + vstinner, eric.araujo, dstufft
2017-04-28 12:17:50vstinnersetmessageid: <1493381870.33.0.670258208626.issue30198@psf.upfronthosting.co.za>
2017-04-28 12:17:50vstinnerlinkissue30198 messages
2017-04-28 12:17:50vstinnercreate