This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: make threading fork-safe
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: cgw, gvanrossum, tim.peters
Priority: normal Keywords: patch

Created on 2000-08-19 04:30 by cgw, last changed 2022-04-10 16:02 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
None cgw, 2000-08-19 04:30 None
Messages (8)
msg33938 - (view) Author: Charles G Waldman (cgw) (Python triager) Date: 2000-08-19 04:30
 
msg33939 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2000-08-25 03:06
Don't check this in yet!

Tim & I had a brainwave. We can share a single mutex for all locks and grab it around a fork, and the lock reinitialization won't be needed.

(We think.)
msg33940 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2000-08-27 17:37
The great idea Tim & I had didn't solve the problem!

The problem has a *parent* spinning. We suspect now that it is a LinuxThreads or kernel bug, since there's otherwise no way that what happens in the child can affect the parent. (Charles' fix only adds code that runs in the child.)

So I've checked in Charles' patch and leave the thread code alone.

Do remember that a long chain of forks will leak a bit -- I see no way around that.
msg33941 - (view) Author: Charles G Waldman (cgw) (Python triager) Date: 2000-08-19 04:46
See http://www.lambdacs.com/newsgroup/FAQ.html#Q120
and "man pthread_atfork" for background.

I don't use a pthread_atfork handler here - the explicit
approach is more portable.  However calling fork() from
C extensions could still cause trouble.

This patch causes the child to create a new interpreter
lock after doing a fork.  It would be nice to deallocate
the old lock with PyThread_free_lock, but this does some
unwanted error-checking in addition to deallocating the
lock.  So I waste a little memory instead.  To really do
this cleanly one could add a new PyThread_reset_lock function to all the thread_*.h files, and use that instead.

msg33942 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2000-08-20 01:51
Assigned to me.  I'll discuss it with Guido too.  So far 2e've gotten 3 reports from people who don't see failures in assorted test cases anymore, and no reports of remaining failures.
msg33943 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2000-08-24 18:02
Accepted, and assigned to Jeremy for checkin.
We may (or may not) want to bulletproof more locks for 2.0b1, but this has certainly helped so far and done no harm.  Jeremy, I don't feel comfortable trying to check in the change myself, as I can't run test_fork1 (or any other fork test) on Windows.
msg33944 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2000-08-24 18:19
Accepted, and assigned to Jeremy for checkin.
We may (or may not) want to bulletproof more locks for 2.0b1, but this has certainly helped so far and done no harm.  Jeremy, I don't feel comfortable trying to check in the change myself, as I can't run test_fork1 (or any other fork test) on Windows.
msg33945 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2000-08-25 04:09
Now that I've had a chance to drive recklessly, suck down a Coke, and chain-smoke two butts in peace <wink>, I'm certain that the "one mutex" idea will work, and satisfied that the extra contention it may cause is pthread's own damn fault for presuming to kill threads in the child while they're in an unknown state.  Full mutex ahead!
History
Date User Action Args
2022-04-10 16:02:18adminsetgithub: 32958
2000-08-19 04:30:35cgwcreate