classification
Title: Embedded python thread crashes
Type: Stage:
Components: Windows Versions:
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Embedded python thread crashes
View: 1193099
Assigned To: Nosy List: amaury.forgeotdarc, nnorwitz, ugodiggi
Priority: normal Keywords:

Created on 2005-04-30 19:03 by ugodiggi, last changed 2020-03-26 23:04 by vstinner. This issue is now closed.

Messages (7)
msg25190 - (view) Author: ugodiggi (ugodiggi) Date: 2005-04-30 19:03
The following code crashes about 1/3 of the times. 

My platform is Python 2.4.1 on WXP (I tried the release
version from the msi and the debug version built by me). 
I can reproduce the same behavior on another wxp
system, under python 2.4. 

The crash happens (in the python thread) while the main
thread is in Py_Finalize. 
I traced the crash to _Py_ForgetReference(op) in
object.c at line 1847, where I have op->_ob_prev == NULL. 

The open file seems to be the issue, since if I remove
all the references to the file I cannot get the program
to crash.

Cheers and ciao 

Ugo 

////////////////////////// TestPyThreads.cpp
////////////////////////// 
#include <windows.h> // Sleep
#include "Python.h" 

int main() 
{ 
        PyEval_InitThreads(); 
        Py_Initialize(); 
        PyGILState_STATE main_restore_state =
PyGILState_UNLOCKED; 
        PyGILState_Release(main_restore_state); 

        // start the thread 
        { 
                PyGILState_STATE state =
PyGILState_Ensure(); 
                int trash = PyRun_SimpleString( 
                                "import thread\n" 
                                "import time\n" 
                                "def foo():\n" 
                                "  f =
open('pippo.out', 'w', 0)\n" 
                                "  i = 0;\n" 
                                "  while 1:\n" 
                                "    f.write('%d\\n'%i)\n" 
                                "    time.sleep(0.01)\n" 
                                "    i += 1\n" 
                                "t =
thread.start_new_thread(foo, ())\n" 
                                ); 
                PyGILState_Release(state); 
        } 

        // wait 300 ms 
        Sleep(300); 

        PyGILState_Ensure(); 
        Py_Finalize(); 
        return 0; 
} 
 
msg25191 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-04 05:37
Logged In: YES 
user_id=33168

I can't reproduce on gentoo linux (amd64) with a debug
build.  I played with different values for sleep.  BTW, it
would be better if you attached the code as a file, since
formatting is lost.

Can you try to debug this problem?
Can you attach info from the debugger?
msg25192 - (view) Author: ugodiggi (ugodiggi) Date: 2005-10-05 18:39
Logged In: YES 
user_id=1269908

I would guess that the problem is going to be windows 
specific?
I tested similar problem on both the debug and release build,
and I did try to get some comment on comp.lang.python
i.e.
http://mail.python.org/pipermail/python-list/2005-
April/278404.html

my comment on the debugging was:
> The crash happens while the main thread is in Py_Finalize.
> I traced the crash to _Py_ForgetReference(op) in object.c 
at line 1847,
> where I have op->_ob_prev == NULL.

I did not look at the python code long enough to follow it.
I will try to reproduce tonight and send out the stack - for 
whatever reason I'm not finding it any longer.
msg25193 - (view) Author: ugodiggi (ugodiggi) Date: 2005-10-06 07:29
Logged In: YES 
user_id=1269908

Something changed on the computer I'm testing on, and the 
crash happens way less often (say 1/10 of the times in debug 
mode, and very rarely in Release mode).

The compiler is MSVC7.1 on WXP, using Python 241

This is the stack of the main thread

 	7ffe0304()	
 	ntdll.dll!77f5b5d4() 	
 	kernel32.dll!77e7a683() 	
 	msvcr71d.dll!_close_lk(int fh=0x00000003)  Line 115 + 
0x4a	C
 	msvcr71d.dll!_close(int fh=0x00000003)  Line 60 + 0x9	
C
 	msvcr71d.dll!_fclose_lk(_iobuf * str=0x1027c898)  Line 
127 + 0xc	C
 	msvcr71d.dll!fclose(_iobuf * stream=0x1027c898)  Line 
58 + 0x9	C
>	python24_d.dll!file_dealloc(PyFileObject * 
f=0x00919ec8)  Line 308 + 0xd	C
 	python24_d.dll!_Py_Dealloc(_object * op=0x00919ec8)  
Line 1870 + 0x7	C
 	python24_d.dll!frame_dealloc(_frame * f=0x00972960)  
Line 394 + 0x67	C
 	python24_d.dll!_Py_Dealloc(_object * op=0x00972960)  
Line 1870 + 0x7	C
 	python24_d.dll!PyThreadState_Clear(_ts * 
tstate=0x00892c88)  Line 200 + 0x6c	C
 	python24_d.dll!PyInterpreterState_Clear(_is * 
interp=0x00894f48)  Line 93 + 0x9	C
 	python24_d.dll!Py_Finalize()  Line 401 + 0x9	C
 	TestPyThreads.exe!main()  Line 33 + 0x8	C++
 	TestPyThreads.exe!mainCRTStartup()  Line 259 + 0x19	
C
 	kernel32.dll!77e8141a()

and this is the stack of the other thread (_threadstart)
(the crash is here, where "if (frame->f_exc_type != NULL) " 
gets called with frame == NULL)
>	python24_d.dll!reset_exc_info(_ts * tstate=0x00892c88)  
Line 2861 + 0x3	C
 	python24_d.dll!PyEval_EvalFrame(_frame * 
f=0x00972960)  Line 2490 + 0x9	C
 	python24_d.dll!PyEval_EvalCodeEx(PyCodeObject * 
co=0x00943ad0, _object * globals=0x008e3230, _object * 
locals=0x00000000, _object * * args=0x008c104c, int 
argcount=0x00000000, _object * * kws=0x00000000, int 
kwcount=0x00000000, _object * * defs=0x00000000, int 
defcount=0x00000000, _object * closure=0x00000000)  Line 
2730 + 0x9	C
 	python24_d.dll!function_call(_object * func=0x009431f0, 
_object * arg=0x008c1038, _object * kw=0x00000000)  Line 
553 + 0x40	C
 	python24_d.dll!PyObject_Call(_object * 
func=0x009431f0, _object * arg=0x008c1038, _object * 
kw=0x00000000)  Line 1751 + 0xf	C
 	python24_d.dll!PyEval_CallObjectWithKeywords(_object 
* func=0x009431f0, _object * arg=0x008c1038, _object * 
kw=0x00000000)  Line 3419 + 0x11	C
 	python24_d.dll!t_bootstrap(void * boot_raw=0x008c8148) 
 Line 433 + 0x1a	C
 	python24_d.dll!bootstrap(void * call=0x0012f694)  Line 
166 + 0x7	C
 	msvcr71d.dll!_threadstart(void * ptd=0x009101a0)  Line 
196 + 0xd	C
 	kernel32.dll!77e7d28e()

as you can see, the crash is also happening at a different 
place than what I had observed before, but I cannot find the 
stack that I had in that case.
msg25194 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-11 03:52
Logged In: YES 
user_id=33168

Perhaps this is caused by different runtime DLLs.  Here's
what I googled for:  site:mail.python.org windows python
runtime dll crash

See http://python.org/sf/1003535

Can you try to research more and see if you can come up with
anything?  (I don't run windows, so I can't test anything.)
 Do you get this problem with the version built from
python.org or only with the version you built?
msg25195 - (view) Author: ugodiggi (ugodiggi) Date: 2005-10-28 05:19
Logged In: YES 
user_id=1269908

I'm sorry I've been so late in answering.
I don't have the time to investigate on the short term, and I'm 
very unfamiliar with python internals.

I did do a fair amount of testing when I originally posted the 
item, including running with the 2.4.0 and 2.4.1 downloaded 
from python.org and 2.4.1 built by me in release and debug 
mode. I also tried on a different windows machine, running XP 
and 2.4.?.
All were showing the same problem.

I've followed the google thread and I don't think it's the same 
problem. 
My best guess is that this is caused by a leaked open file 
handle, that does not get closed correctly in the Py_Finalize 
call, but I'm not quite sure on how to follow this in the 
debugger.

msg62314 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-02-12 12:54
This is a duplicate of issue1856: while a thread is sleeping,
Py_Finalize() deallocates the thread's frame. During deallocation the
thread wakes up and boom.
History
Date User Action Args
2020-03-26 23:04:48vstinnersetsuperseder: Embedded python thread crashes
2020-03-26 23:04:48vstinnerlinkissue1193099 superseder
2008-02-12 12:54:56amaury.forgeotdarcsetstatus: open -> closed
nosy: + amaury.forgeotdarc
resolution: duplicate
messages: + msg62314
2005-04-30 19:03:33ugodiggicreate