Author sam-s
Recipients sam-s
Date 2020-02-12.17:21:03
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1581528064.43.0.461449541499.issue39617@roundup.psfhosted.org>
In-reply-to
Content
The number of workers (max_workers) I want to use often depends on the server load.
Imagine this scenario: I have 64 CPUs and I need to run 200 processes.
However, others are using the server too, so currently loadavg is 50, thus I will set `max_workers` to (say) 20. 
But 2 hours later when those 20 processes are done, loadavg is now 0 (because the 50 processes run by my colleagues are done too), so I want to increase the pool size max_workers to 70.
It would be nice if it were possible to adjust the pool size depending on the server loadavg when a worker is started.
Basically, the intent is maintaining a stable load average and full resource utilization.
History
Date User Action Args
2020-02-12 17:21:04sam-ssetrecipients: + sam-s
2020-02-12 17:21:04sam-ssetmessageid: <1581528064.43.0.461449541499.issue39617@roundup.psfhosted.org>
2020-02-12 17:21:04sam-slinkissue39617 messages
2020-02-12 17:21:03sam-screate