Author jtaylor
Recipients jtaylor
Date 2015-02-26.14:30:53
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1424961054.13.0.0830068196807.issue23530@psf.upfronthosting.co.za>
In-reply-to
Content
multiprocessing.cpu_count and os.cpu_count which are often used to determine how many processes one can run in parallel do not respect the cpuset which may limit the process to only a subset of online cpus leading to heavy oversubscription in e.g. containerized environments:

$ taskset -c 0 python3.4 -c 'import multiprocessing; print(multiprocessing.cpu_count())'
32
$ taskset -c 0 python3.4 -c 'import os; print(os.cpu_count())'
32

While the correct result here should be 1.

This requires programs to have to use less portable methods like forking to gnu nproc or having to read the /proc filesystem.

Having a keyword argument to switch between online and available cpus would be fine too.
History
Date User Action Args
2015-02-26 14:30:54jtaylorsetrecipients: + jtaylor
2015-02-26 14:30:54jtaylorsetmessageid: <1424961054.13.0.0830068196807.issue23530@psf.upfronthosting.co.za>
2015-02-26 14:30:54jtaylorlinkissue23530 messages
2015-02-26 14:30:53jtaylorcreate