New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hang on interpreter shutdown if daemon thread prints to stdout #67498
Comments
A script spawning a single daemon thread calling print() in a loop (like the attached) will usually hang on shutdown in Python 3.4.2 and hg rev 8d802fb6ae32. Attaching gdb at that point shows the following: (gdb) thread apply all bt Thread 1 (Thread 0x7fd927d58700 (LWP 30274)): The daemon thread has exited, and the main thread hangs trying to flush stdout. I haven't fully tracked down what happens here, but I think it's this:
If that is what happens, I don't really see how to fix it (it's an example of daemon threads not releasing their resources, which the documentation warns about). But it's obviously unfortunate if merely writing to stdout/err is such a resource. |
Your analysis sounds right to me. |
A possible solution is to tweak the IO lock acquire on shutdown: only wait for a grace period instead of waiting indefinitely. Attached patch demonstrates the solution (and solves the deadlock issue in your script). |
(note that trying to print a warning or raise an error only makes things worse if the stream subject to deadlock is stderr) |
Attached patch is stricter and dumps a fatal error after the grace period. |
I'm afraid I don't have real-world code I can confirm is fixed, since I reported this on behalf of someone on irc in #python. I think I'd prefer the Py_FatalError version of the patch. It's definitely possible to see writes to stdout/stderr at this time. If I read the first version right, it'll succeed the ENTER_BUFFERED without actually grabbing the lock, which seems problematic. But failing ENTER_BUFFERED is probably also more problematic, as it will probably lead to another exception that Python'll try to write to stderr, which fails the same way... If stdout/stderr are in a broken state during shutdown, Python is probably better off calling Py_FatalError rather than intermittently discarding messages. |
Thank you for the feedback. Yes, I now also think the fatal error is the preferrable way. I'll see if I can add a reliable test. |
Here is a patch with tests. |
New changeset 2c53a5302058 by Antoine Pitrou in branch '3.4': New changeset 8a3da91737cf by Antoine Pitrou in branch 'default': |
Fix pushed! closing. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: