This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author neologix
Recipients bobbyi, gregory.p.smith, neologix, pitrou
Date 2011-05-04.05:56:18
SpamBayes Score 2.4535207e-11
Marked as misclassified No
Message-id <BANLkTikzeM_Y5xodqeDUbOBuJ3qQU4hOnA@mail.gmail.com>
In-reply-to <1304460331.3567.18.camel@localhost.localdomain>
Content
> Yes, we would need to keep track of the thread id and process id inside
> the lock. We also need a global variable of the main thread id after
> fork, and a per-lock "taken" flag.
>
> Synopsis:
>
>    def _reinit_if_needed(self):
>        # Call this before each acquire() or release()
>        if self.pid != getpid():
>            sem_init(self.sem, 0, 1)
>            if self.taken:
>                if self.tid == main_thread_id_after_fork:
>                    # Lock was taken in forked thread, re-take it
>                    sem_wait(self.sem)
>                else:
>                    # It's now released
>                    self.taken = False
>            self.pid = getpid()
>            self.tid = current_thread_id()
>

A couple remarks:
- with linuxthreads, different threads within the same process have
the same PID - it may be true for other implementations - so this
would lead to spurious reinitializations
- what's current_thread_id ? If it's thread_get_ident (pthread_self),
since TID is not guaranteed to be inherited across fork, this won't
work
- calling getpid at every acquire/release is expensive, even though
it's a trivial syscall (it'll have to measured though)
- imagine the following happens:

P1

lock.acquire()
fork()    ->       P2
                       start_new_thread T2
                       T1           T2
                                        lock.acquire()

The acquisition of lock by T2 will cause lock's reinitialization: what
happens to the lock wait queue ? who owns the lock ?
That why I don't think we can delay the reinitialization of locks, but
I could be wrong.

> Well, I fail to understand how that idiom can help us. We're not a
> self-contained application, we're a whole programming language.
> Calling fork() only when no lock is held is unworkable (for example, we
> use locks around buffered I/O objects).

Yes, but in that case, you don't have to reacquire the locks after fork.
In the deadlock you experienced above, the thread that forked wasn't
the one in the I/O code, so the corresponding lock can be
re-initialized anyway, since the thread in the I/O code at that time
won't exist after fork.
And it's true with every lock in the library code: they're only held
in short critical sections (typically acquired when entering a
function and released when leaving), and since it's not the threads
inside those libraries that fork, all those locks can simply be
reinitialized on fork, without having the reacquire them.
History
Date User Action Args
2011-05-04 05:56:19neologixsetrecipients: + neologix, gregory.p.smith, pitrou, bobbyi
2011-05-04 05:56:18neologixlinkissue6721 messages
2011-05-04 05:56:18neologixcreate