New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiprocessing not timely flushing stack trace to stderr #69136
Comments
Related to but not the same than https://bugs.python.org/issue13812. Try this:
It won't show the ZeroDivisionError until you keyboard interrupt the g() process or wait for it to end. This is because _exit_function will join every active non-daemon child, which happens before printing and flushing the error. IMO the exception should be shown before joining children, since keeping the error silent is asking for trouble. |
One possible fix to multiprocessing/process.py: < try:
274a272
This removes the try/finally pair around self.run() and calls _exit_function in the last finally clause. It doesn't honour the original control flow, as this last clause is even executed when the flow is aborted before reaching self.run(). That said, I can't see any particular reason to prefer the original flow. |
If I understand your motivations correctly, I'd restate them as: you want a mechanism for being immediately notified of an exception in a parent process without waiting on any child processes of that parent to finish and furthermore propose this should be the default behavior. Quick side note: As the docs explain and as the code you propose modifying in Lib/multiprocessing/process.py shows, a Process is designed to track its live children. Interrupting or otherwise destroying a parent process's ability to track its children even after experiencing an exception in its target function would not be desirable. I think we agree that the current behavior you describe is not catastrophic in any sense -- the exception occurs, the process finishes waiting on its children, and that exception bubbles up at the end -- nothing is lost or suppressed. This becomes a question of when should a calling routine be notified of such an exception (happening in what I was just calling a parent process): immediately, if we ask for it, or when it's otherwise done? Different use cases motivate different answers to this question. The current behavior satisfies many common use cases and thankfully it is not difficult to implement the other behaviors in your own code. It can start with a try-except inside the example function 'f' where the except clause's code takes some appropriate action right away rather than waiting on the children, if that is what is desired. def f():
try:
p = mp.Process(target=g)
p.start()
1/0
p.join()
except Exception as e:
# Take action, sound the alarm, call out the guards, notify the BDFL!
raise e I'm not sure I see the relation to bpo-13812 which is concerned with traceback text that was sent to stderr but surprisingly wasn't getting flushed to stderr. This issue has no such problem with flushes on stderr -- we are not waiting on stderr here, we are waiting on the parent to finish waiting on its children before any traceback gets written to stderr. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: