This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: print deadlocks
Type: behavior Stage: patch review
Components: IO Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: notarealdeveloper
Priority: normal Keywords: patch

Created on 2021-12-30 23:26 by notarealdeveloper, last changed 2022-04-11 14:59 by admin.

File name Uploaded Description Edit notarealdeveloper, 2021-12-30 23:26 Minimal example of a deadlock the patch fixes.
Pull Requests
URL Status Linked Edit
PR 30310 open notarealdeveloper, 2021-12-30 23:39
Messages (1)
msg409394 - (view) Author: Jason Wilkes (notarealdeveloper) * Date: 2021-12-30 23:26
Hi there. :)

(First time posting here, so apologies in advance if I'm a goof.)

I recently found a deadlock in the stdout and stderr buffer objects.

Following the dev guide, I

(1) wrote up a patch

(2) checked that it fixes the deadlock (on Linux and OS X)

(3) made sure it doesn't break any of the cpython tests (on Linux and OS X)

But I could use some feedback to make sure the patch isn't doing anything dumb, and to make sure I'm not missing something obvious.

Here's the situation.

The deadlock only occurs in the sinful state of threading while forking. (I'm aware of the reasons why one should never do that.)

But asking newcomers to never mix the two is sort of asking them to know the implementation details of every library they use (these abstractions are leaky). And I like teaching people about python and concurrency, so it would be nice if they didn't have to know all that.

So I've recently developed the ill-advised hobby of trying to run as much code as possible in the sinful state of threading while forking, to try to understand which bits of the interpreter and standard library are responsible for most of the deadlocks, in practice.

All the non-obvious deadlocks I've encountered seemed to be due to one thing, namely stdout and stderr's buffer->lock not having an _at_fork_reinit. So if a thread is printing during a fork, the child process can never print, else deadlock. Standard hazard of threading and forking.

What surprised me was that there didn't seem to be 100s of places in the interpreter where threading and forking don't mix. After fixing this one, all the different thread + fork code I'd been testing worked.

Which makes me wonder if there's a reason this hasn't been fixed yet.

Hence, maybe the patch is doing something dumb.

Anyways, this is sort of a bug report, and sort of a request for comments on how I can make this patch better. Equally cool would be an explanation of why re-initializing that lock is a bad idea to begin with.

Thanks for your patience if you read all that.
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90368
2021-12-30 23:39:44notarealdevelopersetkeywords: + patch
stage: patch review
pull_requests: + pull_request28524
2021-12-30 23:26:14notarealdevelopercreate