This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author dgrunwald
Recipients dgrunwald, paul.moore, steve.dower, tim.golden, zach.ware
Date 2021-01-19.18:24:15
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1611080656.89.0.981627513492.issue42968@roundup.psfhosted.org>
In-reply-to
Content
Running the attached script deadlocks.
Uncommenting the `time.sleep(1)` in the script makes the deadlock disappear.

For context: our application uses multiple child processes (multiprocessing.Process) and uses pipes (multiprocessing.Pipe) to communicate with them.
If one process fails with an error, the main process will kill all other child processes running concurrently.
We noticed that sometimes (non-deterministically), when an error occurs soon after startup, the main process ends up hanging.

We expect that when we pass the writing half of a connection to a child process and close the connection in the main process, that we will receive EOFError if the child process terminates unexpectedly.
But sometimes the EOFError never comes and our application hangs.

I've reduced the problem to the script attached. With the reduced script, the deadlock happens reliably for me.

I've debugged this a bit, and I think this is happening because passing a connection to the process being started involves reduce_pipe_connection() which creates a copy of the handle within the main process.
When the pickled data is unpickled in the child process, it uses DUPLICATE_CLOSE_SOURCE to close the copy in the main process.
But if the pickled data is never unpickled by the child process, the handle ends up being leaked.
Thus the main process itself holds the writing half of the connection open, causing the recv() call on the reading half to block forever.
History
Date User Action Args
2021-01-19 18:24:16dgrunwaldsetrecipients: + dgrunwald, paul.moore, tim.golden, zach.ware, steve.dower
2021-01-19 18:24:16dgrunwaldsetmessageid: <1611080656.89.0.981627513492.issue42968@roundup.psfhosted.org>
2021-01-19 18:24:16dgrunwaldlinkissue42968 messages
2021-01-19 18:24:16dgrunwaldcreate