This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Deadlocks with fork() and multithreading
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, facundobatista, georg.brandl, vanco-buca
Priority: normal Keywords:

Created on 2007-07-16 23:34 by vanco-buca, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg32521 - (view) Author: Vanco Buca (vanco-buca) Date: 2007-07-16 23:34
Encountered a deadlock while executing a subprocess.call("<something>") in a multithreaded environment. After some debugging, found the child process stuck on a lock immediately after the fork, inside the module os's _execvpe() function. 

A Google search found a detailed discussion on this topic in the Python-Dev thread (see http://mail.python.org/pipermail/python-dev/2006-May/064983.html). 

My symptoms are exactly the same as the ones found by Rotem Yaari, and exactly the same fix worked as well (moving the "from errno import ENOENT, ENOTDIR" line to the beginning of the os.py file). 

Some details follow: 
- my code has several threads running, and some of them executes shell commands with subprocess.call(). 
- I'm running on Python 2.4.4, with a Linux 2.6.20 kernel, on a dual core system (uname -a shows: Linux rack1 2.6.18-gentoo-r3 #1 SMP PREEMPT Tue Dec 5 16:40:19 PST 2006 x86_64 AMD Opteron(tm) Processor 246 AuthenticAMD GNU/Linux)
msg57027 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2007-11-01 19:06
I followed the link you provided. All the discussion there ends asking
for a realiable way to test the problem (otherwise, we could be making
more mistakes than solving the problem in the different platforms).

Please provide a test case, so we include it in the regression tests and
know that this won't be broken again.

You should provide the smallest code that causes a deadlock for you.
msg57773 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-23 08:36
It sounds like the importer dead lock problem. These problems are almost
impossible to unit test because they are usually race conditions.

I don't see a problem in moving the import to the module name space.
errno is a built-in module and the import isn't costly.
msg57879 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2007-11-27 18:50
Fixed in the trunk, rev 59195.
msg57899 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-11-27 23:56
Will it be backported?
msg59464 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2008-01-07 16:49
Backported, commited in r59823.
History
Date User Action Args
2022-04-11 14:56:25adminsetgithub: 45212
2008-01-07 16:49:46facundobatistasetstatus: open -> closed
resolution: fixed
messages: + msg59464
2007-11-27 23:56:10georg.brandlsetnosy: + georg.brandl
messages: + msg57899
2007-11-27 18:50:42facundobatistasetmessages: + msg57879
2007-11-23 08:36:52christian.heimessetnosy: + christian.heimes
messages: + msg57773
2007-11-01 19:06:01facundobatistasetassignee: facundobatista ->
messages: + msg57027
2007-10-23 20:24:45facundobatistasetassignee: facundobatista
nosy: + facundobatista
2007-07-16 23:34:28vanco-bucacreate