Message313401
That stackoverflow thread points to the GNU coreutils 'nproc', which is an interesting compendium of knowledge about this problem.
It looks like their algorithm is roughly:
1. Determine how many CPUs *could* this program access, by going down a list of possible options and using the first that works: pthread_getaffinity_np -> sched_getaffinity -> GetProcessAffinityMask -> sysconf(_SC_NUMPROCESSORS_ONLN) -> some arcane stuff specific to HP-UX, IRIX, etc.
2. Parse the OMP_NUM_THREADS and OMP_THREAD_LIMIT envvars (this is not quite trivial, there's some handling of whitespace and commas and references to the OMP spec)
3. If OMP_NUM_THREADS is set, return min(OMP_NUM_THREADS, OMP_THREAD_LIMIT). Otherwise, return min(available_processors, OMP_THREAD_LIMIT).
Step (1) handles both the old affinity APIs, and also the cpuset system that docker uses. (From cpuset(7): "Cpusets are integrated with the sched_setaffinity(2) scheduling affinity mechanism".) Step (2) relies on the quasi-standard OMP_* envvars to let you choose something explicitly.
The PBS Pro docs say that they set both NPROCS and OMP_NUM_THREADS. See section 6.1.7 of https://pbsworks.com/pdfs/PBSUserGuide14.2.pdf
So this seems like a reasonable heuristic approach to me. |
|
Date |
User |
Action |
Args |
2018-03-07 22:26:16 | njs | set | recipients:
+ njs, pitrou, davin, Matthew Rocklin, Matt Harvey, M J Harvey |
2018-03-07 22:26:16 | njs | set | messageid: <1520461576.42.0.467229070634.issue32986@psf.upfronthosting.co.za> |
2018-03-07 22:26:16 | njs | link | issue32986 messages |
2018-03-07 22:26:16 | njs | create | |
|