This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author njs
Recipients asvetlov, holger+lp, njs, vstinner, yselivanov
Date 2018-02-05.22:26:18
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1517869578.51.0.467229070634.issue32776@psf.upfronthosting.co.za>
In-reply-to
Content
There's two separate issues here: the warning spew because asyncio's internal signal handling code starts losing signals when they arrive too quickly, and the way the child reaping loop polls all the children on every SIGCHLD, which makes reaping N children an O(N**2) operation.

The warning spew is inherent in asyncio's current signal handling design. (BTW, Victor, here's an answer to your question in https://bugs.python.org/issue30050#msg291533 about whether this overflow can happen in real life :-).) It's not *terribly* harmful – losing some SIGCHLDs among thousands doesn't matter. It does mean that you could lose other signals if they happen to arrive at the same time, e.g. SIGINT and SIGTERM are probably ignored while this is happening.

For the O(N**2) issue, I think you can work around it by using some incantation involving set_child_watcher and the FastChildWatcher class. These aren't documented (maybe someone should document them!) and I don't know the exact details off the top of my head, but it is a public interface for making your child reaping O(N). (This uses the wait(-1, ...) trick. Unfortunately there are arcane technical limitations with signalfd and si_pid that mean they can't solve this problem. Something like CLONE_FD would help, but that patch was never merged into Linux.)

You might also consider switching to something like uvloop, which has a more robust event loop implementation underneath. I haven't checked, but I'm pretty sure it'd fix the signal buffer overflow issue, and I doubt they use an O(N**2) child reaping algorithm.
History
Date User Action Args
2018-02-05 22:26:18njssetrecipients: + njs, vstinner, asvetlov, yselivanov, holger+lp
2018-02-05 22:26:18njssetmessageid: <1517869578.51.0.467229070634.issue32776@psf.upfronthosting.co.za>
2018-02-05 22:26:18njslinkissue32776 messages
2018-02-05 22:26:18njscreate