classification
Title: thread + import => crashes?
Type: Stage:
Components: Documentation Versions: Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Rhamphoryncus, andyshorts, christian.heimes, forest, georg.brandl, ncoghlan, ocean-city
Priority: high Keywords:

Created on 2007-05-17 13:08 by ocean-city, last changed 2008-12-12 02:15 by forest. This issue is now closed.

Files
File name Uploaded Description Edit
bug.py ocean-city, 2007-05-17 13:08 The script to reproduce error
bug.zip ocean-city, 2007-05-17 14:49 The script to reproduce error (version2)
Messages (17)
msg32028 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2007-05-17 13:08
I got some experience of crash on Tkinter, so I minimized triger code. (I'll attach it as 'bug.py')
Same error happens on released python 2.5.1 binary, trunk built with VC6, and release25-maint built with VC6. (Not every time this error occurs, but frequently)

# Here is error message.

S:\python\tkinter>bug.py
Traceback (most recent call last):
  File "S:\python\tkinter\bug.py", line 13, in <module>
    raise RuntimeError()
RuntimeError
Fatal Python error: PyImport_GetModuleDict: no module dictionary!

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

# Here is stack trace. (Win2000SP4 + VC6)

NTDLL! 77f9193c()
PyImport_GetModuleDict() line 361 + 10 bytes
import_submodule(_object * 0x1e3148b8 __Py_NoneStruct, char * 0x013fe440, char * 0x013fe440) line 2357 + 5 bytes
load_next(_object * 0x1e3148b8 __Py_NoneStruct, _object * 0x1e3148b8 __Py_NoneStruct, char * * 0x013fe554, char * 0x013fe440, int * 0x013fe43c) line 2216 + 17 bytes
import_module_level(char * 0x00000000, _object * 0x008e6620, _object * 0x1e3148b8 __Py_NoneStruct, _object * 0x1e3148b8 __Py_NoneStruct, int -1) line 1997 + 35 bytes
PyImport_ImportModuleLevel(char * 0x009408cc, _object * 0x008e6620, _object * 0x1e3148b8 __Py_NoneStruct, _object * 0x1e3148b8 __Py_NoneStruct, int -1) line 2068 + 25 bytes
builtin___import__(_object * 0x00000000, _object * 0x00b0aaa0, _object * 0x00000000) line 48 + 25 bytes
PyCFunction_Call(_object * 0x008cf478, _object * 0x00b0aaa0, _object * 0x00000000) line 77 + 15 bytes
PyObject_Call(_object * 0x008cf478, _object * 0x00b0aaa0, _object * 0x00000000) line 1860 + 15 bytes
PyEval_CallObjectWithKeywords(_object * 0x008cf478, _object * 0x00b0aaa0, _object * 0x00000000) line 3434
PyEval_EvalFrameEx(_frame * 0x00a5d3e8, int 107) line 2065
fast_function(_object * 0x00000000, _object * * * 0x013fef00, int 1, int 1, int 10152296) line 3651
call_function(_object * * * 0x013fef00, int 0) line 3585 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00a706b8, int 131) line 2269
PyEval_EvalCodeEx(PyCodeObject * 0x009e80e8, _object * 0x00a706b8, _object * 0x00000002, _object * * 0x00ac86a0, int 2, _object * * 0x00ac86a8, int 0, _object * * 0x00a1ccfc, int 1, _object * 0x00000000) line 2831 + 11 bytes
fast_function(_object * 0x00000001, _object * * * 0x013ff418, int 2, int 2, int 0) line 3663 + 53 bytes
call_function(_object * * * 0x013ff418, int 0) line 3585 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00ac8528, int 131) line 2269
PyEval_EvalCodeEx(PyCodeObject * 0x009e4c88, _object * 0x00ac8528, _object * 0x00000002, _object * * 0x00a5b590, int 2, _object * * 0x00a5b598, int 0, _object * * 0x00a1cc1c, int 1, _object * 0x00000000) line 2831 + 11 bytes
fast_function(_object * 0x00000001, _object * * * 0x013ff930, int 2, int 2, int 0) line 3663 + 53 bytes
call_function(_object * * * 0x013ff930, int 0) line 3585 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00a5b440, int 131) line 2269
PyEval_EvalCodeEx(PyCodeObject * 0x009e44a8, _object * 0x00a5b440, _object * 0x00000001, _object * * 0x00a5b224, int 1, _object * * 0x00a5b228, int 0, _object * * 0x00a1e9cc, int 2, _object * 0x00000000) line 2831 + 11 bytes
fast_function(_object * 0x00000002, _object * * * 0x013ffe48, int 1, int 1, int 0) line 3663 + 53 bytes
call_function(_object * * * 0x013ffe48, int 0) line 3585 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00a5b0e0, int 131) line 2269
PyEval_EvalCodeEx(PyCodeObject * 0x009d5ce8, _object * 0x00a5b0e0, _object * 0x00000000, _object * * 0x008c104c, int 0, _object * * 0x00000000, int 0, _object * * 0x00000000, int 0, _object * 0x00000000) line 2831 + 11 bytes
function_call(_object * 0x00a2b140, _object * 0x008c1038, _object * 0x00000000) line 522 + 64 bytes
PyObject_Call(_object * 0x00a2b140, _object * 0x008c1038, _object * 0x00000000) line 1860 + 15 bytes
PyEval_CallObjectWithKeywords(_object * 0x00a2b140, _object * 0x008c1038, _object * 0x00000000) line 3434
t_bootstrap(void * 0x008c8508) line 425 + 26 bytes
bootstrap(void * 0x0022f8cc) line 179 + 7 bytes
_threadstart(void * 0x009bff78) line 187 + 13 bytes
KERNEL32! 77e5b396()
msg32029 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2007-05-17 14:49
Sorry, Tkinter and urllib is not source of problem. I changed summary to "thread + import => crashes?".

# I'll attach reproducable script as "bug.zip". please run main.py

Probabry "import" in another thread causes crash, but I'll investigate more.
File Added: bug.zip
msg32030 - (view) Author: AndyShorts (andyshorts) Date: 2007-06-19 23:40
I have had a look at this using the latest trunk code built using VisualStudio 2005 on Windows XP SP2.

I modified the source scripts from bug2.zip so that the code was wrapped in a function and added the same code as a function to main.py; I did this so as to see if the difference in calling method would affect it at all, i.e. if by removing the import of sub.py it would make the issue any better or worse.

Running it in the debugger and modifying the debug output from the engine to include Win32 thread identifiers seems to suggest that the main thread is exiting and cleaning up the Python system modules and then the Win32 system is closing the other Python threads which are then trying to access the global Python modules and they can't find them and they exit. [Though as a /total/ newbie to Python I am only about 40-50% confident of this] And as to be expected the error does not happen all the time - I can run the same script and maybe get the error as listed maybe about 60-70% of the time.

Interestingly in main.c there is a call to a function called "WaitForThreadShutdown" - but sadly it only looks to see if the Python threading module has been imported. I think extending this function to include the Python thread module might be the way to go - but I really don't know and I certainly haven't got to grips with the Python source yet.
msg60158 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-19 13:52
For 2.6 I've fixed several places where C code was importing a Python
module in a non-threadsafe way. In general a thread start must never be
caused by an import and only the main thread should import modules.
Imports within or caused-by a thread other than the main thread isn't safe.
msg60159 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-19 13:55
Follow up:

Can somebody please document the possible issue with threads and imports
in large, friendly letters?
msg62842 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-02-23 23:25
Documented in both thread and threading docs in r61029.
msg62879 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 06:18
Reopening - I disagree with the assertion that this isn't expected to work.
msg62880 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:04
Hmm, I can't reproduce any failure using the supplied zip file, even if
I increase the number of threads to 382 (if I set it any higher,
start_new_thread starts to fail). That's with both 2.5.1 and 2.6 SVN.

I'll try again with the original tkinter/urllib script.
msg62881 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:10
Nope, can't reproduce it with the original tkinter/urllib script either
(again, Ubuntu's 2.5.1 release and SVN 2.6, with the thread count set at
350).

Could it be a windows specific problem? Or require a non-hyperthreaded CPU?
msg62882 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:15
Ah, never mind - just needed to run the test a few more times to get it
to fail.
msg62884 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:40
Ahah, found the real restrictions - while it turns out the OP's
demonstration code is indeed broken, I think the warning Georg added to
the docs based on Christian's comment above is *far* too strong (hence
my somewhat annoyed message on the checkins list ;).

The actual restriction is given in Py_Finalize(): all pending imports
must be given a chance to finish before the interpreter is torn down. No
new imports should be started after that point.

The threading module takes care of this for you automatically by
registering a system shut down handler that will call the join() method
of all non-daemon threads before the interpreter starts to be torn down.
As daemon threads aren't included in this, all of their imports must be
completed before the thread is switched to daemon mode.

With the thread module, a lot more is left up to the caller (one of the
main reasons this module is being renamed to _thread in Py3k). You can
make the warnings against using *this* module directly as strong as you
want - doing so really can get you in big trouble.

As for the exact nature of the bug in the OP's demonstration code: it
takes no measures to ensure that all imports are complete before the
interpreter is shut down (and in fact deliberately goes out of its way
to break the restriction). While the approach Christian suggests
(perform all your imports from the main thread) is certainly one way to
obey the restriction, it is far from the only way (e.g. use the
threading module, and have any daemon threads only flag themselves as
such once they finish their imports).
msg62885 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:50
Correction to previous comment: you can't safely perform imports from
daemon threads at all, as you have to set their daemon status before you
start them.

So, to be able to safely import from a thread other than the main thread:
- it must be created by the threading module, or otherwise guaranteed to
be terminated when the interpreter exits
- if created through the threading module, it must not be set to daemon mode
msg62886 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-24 07:55
And one other restriction: the thread performing the import must not be
spawned as a side effect of importing a module (as this will deadlock
the interpreter)

The effect of disobeying the two restrictions in my previous comment
will be intermittent crashes at shutdown, such as those experienced by
the original poster of this bug report.
msg63011 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-25 22:47
Proposed wording for a warning pointing that threads and imports can get
you in trouble. It might be better to squirrel off into a new subsection
on threaded imports, rather than giving it too much prominence (since
the recommended approach to creating subthreads should 'just work' -
it's only daemon threads and using the thread module directly that can
get you in trouble):

"""While the import machinery is thread safe, there are two key
restrictions on threaded imports due to inherent limitations in the way
that thread safety is provided.
Firstly, other than in the main module, an import should not have the
side effect of spawning a new thread and then waiting for that thread in
any way. Failing to abide by this restriction can lead to a deadlock if
the spawned thread directly or indirectly attempts to import a module.
Secondly, all import attempts must be completed before the interpreter
starts shutting itself down. This can be most easily achieved by only
performing imports from non-daemon threads created through the threading
module. Daemon threads and threads created directly with the thread
module will require some other form of synchronization to ensure they do
not attempt imports after system shutdown has commenced. Failure to
abide by this restriction will lead to intermittent exceptions and
crashes during interpreter shutdown (as the late imports attempt to
access machinery which is no longer in a valid state)."""

(If we were going to actually change anything in the code, it would be
to ensure that breaking the second restriction could only lead to
exceptions rather than crashes - I suspect that would be pretty messy
though, as there are a bunch of PyImport_* calls that currently can't
fail that would start having to be checked for error returns)
msg63012 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-02-25 22:49
(oh, one thing - the 'crashes' mentioned in my proposed wording are due
to explicit calls to PyErr_FatalError rather than a segfault or anything
like that. The one specifically reported by the OP is due to sys.modules
being gone by the time an import is attempted)
msg63498 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-03-13 07:22
Added your remarks to threading docs in r61365.
msg67147 - (view) Author: Adam Olsen (Rhamphoryncus) Date: 2008-05-21 05:55
The patch for issue 1856 should fix the potential crash, so we could
eliminate that scary blurb from the docs.
History
Date User Action Args
2008-12-12 02:15:54forestsetnosy: + forest
2008-05-21 05:56:04Rhamphoryncussetnosy: + Rhamphoryncus
messages: + msg67147
2008-03-13 07:22:43georg.brandlsetstatus: open -> closed
resolution: fixed
messages: + msg63498
2008-02-25 22:49:22ncoghlansetmessages: + msg63012
2008-02-25 22:47:35ncoghlansetmessages: + msg63011
2008-02-24 07:55:01ncoghlansetmessages: + msg62886
2008-02-24 07:50:21ncoghlansetmessages: + msg62885
2008-02-24 07:40:46ncoghlansetmessages: + msg62884
2008-02-24 07:15:24ncoghlansetmessages: + msg62882
2008-02-24 07:10:31ncoghlansetmessages: + msg62881
2008-02-24 07:04:42ncoghlansetmessages: + msg62880
2008-02-24 06:18:45ncoghlansetstatus: closed -> open
nosy: + ncoghlan
resolution: fixed -> (no value)
messages: + msg62879
2008-02-23 23:25:44georg.brandlsetstatus: open -> closed
resolution: fixed
messages: + msg62842
nosy: + georg.brandl
2008-01-19 13:55:13christian.heimessetmessages: + msg60159
components: + Documentation, - None
2008-01-19 13:52:22christian.heimessetnosy: + christian.heimes
messages: + msg60158
2007-05-17 13:08:39ocean-citycreate