classification
Title: Embedded python reinitialization
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.0
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Karemir, christian.heimes, gvanrossum
Priority: high Keywords:

Created on 2007-10-20 13:35 by Karemir, last changed 2008-02-24 14:12 by christian.heimes. This issue is now closed.

Files
File name Uploaded Description Edit
py3k_reinit.patch christian.heimes, 2007-11-21 04:55
py3k_reinit2.patch christian.heimes, 2007-11-21 08:19
Messages (7)
msg56603 - (view) Author: (Karemir) Date: 2007-10-20 13:35
Hi, 
I'm embedding stackless python in a c++ application under linux. When I 
try to Finalize and Initialize again, I'm getting errors. this is a 
sample code that doesn't work:

int main() {
        Py_Initialize();
        Py_Finalize();
        Py_Initialize();
        Py_Finalize();
}

Second call to Init throws out a nice Segmentation Fault. Here's a 
stack trace:

PyDict_SetItem (op=0x0, key=0x2aaaaab1a4c8, value=0x6bab20) at 
Objects/dictobject.c:616
616             if (!PyDict_Check(op)) {
Current language:  auto; currently c
(gdb) bt
#0  PyDict_SetItem (op=0x0, key=0x2aaaaab1a4c8, value=0x6bab20) at 
Objects/dictobject.c:616
#1  0x0000000000476ac9 in PyDict_SetItemString (v=0x0, 
key=Variable "key" is not available.
) at Objects/dictobject.c:2178
#2  0x00000000004ea77f in PyErr_PrintEx (set_sys_last_vars=1) at 
Python/pythonrun.c:1098
#3  0x00000000004ebfb9 in Py_InitializeEx (install_sigs=1) at 
Python/pythonrun.c:995
#4  0x000000000044a8cc in main () at main.cpp:23

I'm using gcc 3.4.6, and Python 2.5.1 Stackless 3.1b3.
msg57725 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-21 04:01
Python 3.0 suffers from an problem with reinitialization but I can't
reproduce the bug with the release branch of 2.5. Python 2.6 also does
fine but leaks some references.

// reinit_test.c
#include "Python.h"
#define ROUNDS 5

int main(void) {
    int i;
    for (i=0; i < ROUNDS; i++) {
        printf("round %d\n", i+1);
        Py_Initialize();
        Py_Finalize();
    }

    return 0;
}

gcc -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -IInclude -I. -pthread 
-Xlinker -lpthread -ldl  -lutil -lm -export-dynamic -o reinit_test
reinit_test.c libpython2.5.a 

Python 3.0
----------

./reinit_test
round 1
[23832 refs]
round 2
Fatal Python error: Py_Initialize: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/local/lib/python3.0/encodings/__init__.py", line 32, in
<module>
    from . import aliases
ValueError: Cannot encode path item
Aborted

Python 2.6
----------

$ ./reinit_test
round 1
[7349 refs]
round 2
[7393 refs]
round 3
[7428 refs]
round 4
[7463 refs]
round 5
[7498 refs]

Python 2.5
----------
$ ./reinit_test
round 1
[7309 refs]
round 2
[7349 refs]
round 3
[7380 refs]
round 4
[7411 refs]
round 5
[7442 refs]
msg57727 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-21 04:55
The patch solves one issue. It resets the Py_DefaultFileSystemEncoding
to NULL when no default value was given. However the finalization fails
after the 3rd round at 

        if (op == &refchain ||
            op->_ob_prev->_ob_next != op || op->_ob_next->_ob_prev != op)
                Py_FatalError("UNREF invalid object");

$ gdb ./reinit_test
GNU gdb 6.6-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1".
(gdb) run
Starting program: /home/heimes/dev/python/py3k/reinit_test
[Thread debugging using libthread_db enabled]
[New Thread -1210464064 (LWP 8060)]
round 1
[23832 refs]
round 2
[24558 refs]
round 3
Fatal Python error: UNREF invalid object

Program received signal SIGABRT, Aborted.
[Switching to Thread -1210464064 (LWP 8060)]
0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7dc7875 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xb7dc9201 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0x0805d62d in Py_FatalError (msg=0x81790d8 "UNREF invalid object")
at Python/pythonrun.c:1761
#4  0x080aac57 in _Py_ForgetReference (op=0x821d728) at
Objects/object.c:1536
#5  0x080aaca6 in _Py_Dealloc (op=0x821d728) at Objects/object.c:1555
#6  0x080c0f52 in tupledealloc (op=0x82ebc3c) at Objects/tupleobject.c:169
#7  0x080aacb1 in _Py_Dealloc (op=0x82ebc3c) at Objects/object.c:1556
#8  0x080c0f52 in tupledealloc (op=0x83f9154) at Objects/tupleobject.c:169
#9  0x080aacb1 in _Py_Dealloc (op=0x83f9154) at Objects/object.c:1556
#10 0x08083605 in code_dealloc (co=0x83b1bc8) at Objects/codeobject.c:276
#11 0x080aacb1 in _Py_Dealloc (op=0x83b1bc8) at Objects/object.c:1556
#12 0x081691bf in func_dealloc (op=0x83f7924) at Objects/funcobject.c:555
#13 0x080aacb1 in _Py_Dealloc (op=0x83f7924) at Objects/object.c:1556
#14 0x080a272e in PyDict_Clear (op=0x83f6df4) at Objects/dictobject.c:798
#15 0x080a524f in dict_tp_clear (op=0x83f6df4) at Objects/dictobject.c:1793
#16 0x08068c13 in delete_garbage (collectable=0xbf9fa024, old=0x81978e8)
at Modules/gcmodule.c:688
#17 0x0806909f in collect (generation=2) at Modules/gcmodule.c:824
#18 0x080699ea in PyGC_Collect () at Modules/gcmodule.c:1238
#19 0x0805a022 in Py_Finalize () at Python/pythonrun.c:416
#20 0x08059791 in main () at reinit_test.c:9
(gdb)
msg57734 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-21 08:19
I've added some debugging code to _Py_ForgetReference and now I'm
getting this:

$ ./reinit_test
round 1
[23832 refs]
round 2
[24558 refs]
round 3
* ob
object  : <refcnt 0 at 0x821d728>
type    : str
refcount: 0
address : 0x821d728
* op->_ob_prev->_ob_next
object  : <refcnt 0 at 0x821d728>
type    : str
refcount: 0
address : 0x821d728
* op->_ob_next->_ob_prev
object  : buffer(b'')
type    : buffer
refcount: 1
address : 0x826c8c8
* refchain
object  : <refcnt 0 at 0x81a1500>
type    : NULL
refcount: 0
address : 0x81a1500
Fatal Python error: UNREF invalid object
Aborted
msg57738 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-11-21 19:18
Note that the OP was complaining about Stackless. This is not the place
to report issues with that.
msg60172 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-19 14:42
Right, the Python bug tracker is not the place to report Stackless bugs.
But my tests have revealed a bug in Python 3.0. As today multiple
reinitialization still fails.

I suspect a reference bug but I can't proof it.
msg62905 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-02-24 14:12
I've fixed the bug in Python 3.0 a few weeks ago. I'm not sure about the
stackless bug but this is the wrong place to report it. Please contact
the other Christian.
History
Date User Action Args
2008-02-24 14:12:51christian.heimessetstatus: open -> closed
resolution: not a bug
messages: + msg62905
2008-01-19 14:42:04christian.heimessetpriority: low -> high
assignee: gvanrossum ->
messages: + msg60172
versions: - Python 2.6, Python 2.5
2008-01-06 22:29:44adminsetkeywords: - py3k
versions: Python 2.6, Python 2.5, Python 3.0
2007-11-21 19:18:18gvanrossumsetpriority: normal -> low
messages: + msg57738
2007-11-21 08:19:55christian.heimessetfiles: + py3k_reinit2.patch
messages: + msg57734
2007-11-21 04:56:00christian.heimessetfiles: + py3k_reinit.patch
assignee: gvanrossum
messages: + msg57727
nosy: + gvanrossum
2007-11-21 04:01:08christian.heimessetseverity: urgent -> normal
versions: + Python 2.6, Python 3.0
nosy: + christian.heimes
messages: + msg57725
priority: normal
keywords: + py3k
2007-10-20 13:35:09Karemircreate