Message140402
Here is a morning reasoning exercise - please help find the flaws or refine it:
5) Sanitizing worker threads in the multiprocessing module
Sanitizing a worker thread in the context of this problem is to make sure it can not create a state that may deadlock another thread that calls fork(); or in other words fork-safe.
Keep in mind that in Python os.fork() is never called from a POSIX signal handler.
So what are examples of a fork-safe thread?
a) A thread that spins endlessly doing nothing in a C for(;;) loop is safe.
Another thread may call fork() without restrictions.
b) A Python thread that only calls function that do not yield the GIL and that does not acquire locks that are held beyond a Python tick is safe.
An example for such a lock is a critical-section lock acquired by a lower level third party library for the duration of a function call.
Such a lock will be released by the time os.fork() is called because of the GIL.
c) A Python thread that in addition to (2) also acquires a lock that is handled at fork is safe.
d) A Python thread that in addition to (2) and (3) calls function that yield the GIL but while the GIL is released only calls async-signal-safe code.
This is a bit tricky. We know that it is safe for thread A to fork and call async-signal-safe functions regardless of what thread B has been doing, but I do not know that thread A can fork and call non async-signal-safe functions if thread B was only calling async-signal-safe functions.
Nevertheless it makes sense: For example lets assume it isn't true, and that hypothetical thread A forked while thread B was doing the async-signal-safe function safe_foo(), and then thread A called non async-signal-safe function unsafe_bar() and deadlocked.
unsafe_bar() could have deadlocked trying to acquire a lock that was acquired by safe_foo(). But if this is so, then it could also happen the other way around.
Are there other practical possibilities?
Either way, we could double check and white list the async-signal-safe functions we are interested in, in a particular implementation.
e) Socket related functions such as bind() accept() send() and recv(), that Python calls without holding the GIL, are all async-signal-safe.
This means that in principle we can have a fork-safe worker thread for the purpose of communicating with a forked process using a socket. |
|
Date |
User |
Action |
Args |
2011-07-15 11:17:45 | nirai | set | recipients:
+ nirai, gregory.p.smith, pitrou, vstinner, bobbyi, neologix, Giovanni.Bajo, sdaoden, avian |
2011-07-15 11:17:45 | nirai | set | messageid: <1310728665.07.0.090197080132.issue6721@psf.upfronthosting.co.za> |
2011-07-15 11:17:44 | nirai | link | issue6721 messages |
2011-07-15 11:17:43 | nirai | create | |
|