Message292530
Since Python 3.5, distutils is able to build extensions in parallel, nice enhancement! But setup.py of CPython is 2x slower in parallel mode when all modules are already built: (1 sec vs 500 ms).
Building extensions calls newer_group() which calls os.stat() 6,856 times. I wrote a cache for os.stat() but it has no impact on performance.
It seems like threads are fighting to death for the GIL in the os.stat() race...
Attached pull request calls newer_group() before spawning threads in parallel mode, so "setup.py build" takes the same time with and without parallel module, when all extensions are already built.
I didn't measure performance when all extensions must be built. |
|
Date |
User |
Action |
Args |
2017-04-28 12:17:50 | vstinner | set | recipients:
+ vstinner, eric.araujo, dstufft |
2017-04-28 12:17:50 | vstinner | set | messageid: <1493381870.33.0.670258208626.issue30198@psf.upfronthosting.co.za> |
2017-04-28 12:17:50 | vstinner | link | issue30198 messages |
2017-04-28 12:17:50 | vstinner | create | |
|