Message236671
multiprocessing.cpu_count and os.cpu_count which are often used to determine how many processes one can run in parallel do not respect the cpuset which may limit the process to only a subset of online cpus leading to heavy oversubscription in e.g. containerized environments:
$ taskset -c 0 python3.4 -c 'import multiprocessing; print(multiprocessing.cpu_count())'
32
$ taskset -c 0 python3.4 -c 'import os; print(os.cpu_count())'
32
While the correct result here should be 1.
This requires programs to have to use less portable methods like forking to gnu nproc or having to read the /proc filesystem.
Having a keyword argument to switch between online and available cpus would be fine too. |
|
Date |
User |
Action |
Args |
2015-02-26 14:30:54 | jtaylor | set | recipients:
+ jtaylor |
2015-02-26 14:30:54 | jtaylor | set | messageid: <1424961054.13.0.0830068196807.issue23530@psf.upfronthosting.co.za> |
2015-02-26 14:30:54 | jtaylor | link | issue23530 messages |
2015-02-26 14:30:53 | jtaylor | create | |
|