classification
Title: Segfault in stringobject.c
Type: crash Stage:
Components: XML Versions: Python 2.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, farshad, vstinner
Priority: normal Keywords:

Created on 2008-11-19 18:34 by farshad, last changed 2009-03-24 23:37 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
backtrace1 farshad, 2008-11-19 18:34 GDB Backtrace
backtrace2 farshad, 2008-11-19 18:35 GDB Backtrace
Messages (8)
msg76066 - (view) Author: Farshad Khoshkhui (farshad) Date: 2008-11-19 18:34
I'm encountering random segfaults on multiple machines. By examining
core dumps, it's all happening  in stringobject.c (_PyString_Resize or
string_join). 
By using pyframev I figured out it's always happening inside
xmlrpclib.py (/usr/lib/python2.5/xmlrpclib.py (716): dump_struct or
/usr/lib/python2.5/xmlrpclib.py (627): dumps)

I'm using Python 2.5.2 on debian in a threaded environment with heavy
use of xmlrpc.
msg76067 - (view) Author: Farshad Khoshkhui (farshad) Date: 2008-11-19 18:35
Another Backtrace. I have three other core dumps with exact same
backtrace on two different machines.
msg76069 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2008-11-19 19:00
Has nothing to do with ctypes (the package), unassigning.
msg76078 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-11-19 23:15
This is difficult: the backtrace only show plain python operations.
Some hints though:

One backtrace shows a memory corruption in the obmalloc data. This may come from a 
buffer overrun.

You initially selected ctypes in "Components", does this mean that your program 
also use ctypes?
With ctypes it is easy to be caught with a python string converted to a (mutable) 
char* pointer.

For example, on Windows:
>>> import ctypes
>>> c = ctypes.CDLL('msvcrt')
>>> a = "    "
>>> c.strcpy(a, "X" * 50)
50
>>> a
'XXXX'
... and the crash is not far.

Another case of corruption in obmalloc is to try to allocate python objects while 
the GIL is not held. This may happen if you wrote a C function that uses the 
Python API, and call this with ctypes.

In any case, I suggest that you build and a use a debug-enabled version of python 
(configure with --with-pydebug). It catches some errors earlier and sometimes more 
reliably.
msg76091 - (view) Author: Farshad Khoshkhui (farshad) Date: 2008-11-20 07:00
No, there isn't any custom made C extension, nor I'm using ctypes. (It
was a mistake selecting ctypes).
The application is a web service with postgresql backend, so it heavily
uses pyexpat and pygresql in a threaded environment. 
I'll recompile python with pydebug and get back with results.
msg76184 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2008-11-21 14:23
> The application is a web service with postgresql backend, so it
> heavily uses pyexpat and pygresql in a threaded environment.

pyexpat or pygresql is maybe not thread safe. To catch such error, you 
have to use Valgrind. And to use Valgrind, you have to recompile 
Python with the define "Py_USING_MEMORY_DEBUGGER": read 
Misc/valgrind.supp (comments at the top). Then, 
use "valgrind --suppressions=Misc/valgrind.supp <your program> 
<arguments...>". You might also try 
helgrind: "valgrind --tool=helgrind --suppressions=Misc/valgrind.supp 
<your program> <arguments...>".

Note: remember me to translate this article to english!
http://www.haypocalc.com/blog/index.php/2008/08/22/160-deboguer-programme-python-avec-gdb
msg83720 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-03-17 23:59
farshad: Is the bug still open? If yes, can you give more 
informations? If you don't answer, I will have to close this issue 
because it's not possible find the bug with so few informations :-/
msg84124 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-03-24 23:37
There are not enough informations to reproduce the issue or understand 
the problem. Since farshad didn't answer since 4 months, I choose to 
close the bug. Reopen the bug if you have new informations!
History
Date User Action Args
2009-03-24 23:37:22vstinnersetstatus: open -> closed
resolution: not a bug
messages: + msg84124
2009-03-17 23:59:10vstinnersetmessages: + msg83720
2008-11-21 14:23:18vstinnersetnosy: + vstinner
messages: + msg76184
2008-11-20 07:00:11farshadsetmessages: + msg76091
2008-11-19 23:15:03amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg76078
2008-11-19 19:50:59thellersetassignee: theller ->
nosy: - theller
2008-11-19 19:00:31thellersetnosy: theller, farshad
messages: + msg76069
components: - ctypes
2008-11-19 18:35:19farshadsetfiles: + backtrace2
messages: + msg76067
2008-11-19 18:34:29farshadcreate