classification
Title: Segmentation fault in test suite of apsw
Type: Stage: resolved
Components: Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: pitrou Nosy List: Arfrever, benjamin.peterson, georg.brandl, pitrou, rogerbinns, skrah
Priority: normal Keywords:

Created on 2012-09-25 09:40 by Arfrever, last changed 2012-09-26 07:47 by rogerbinns. This issue is now closed.

Messages (9)
msg171236 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-09-25 09:40
The fix for issue #13992 introduced segmentation fault in test suite of apsw (http://code.google.com/p/apsw/) (with Python 2.7, 3.2 and 3.3).

$ python2.7 setup.py build
running build
running build_ext
SQLite: Using system sqlite include/libraries
building 'apsw' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
x86_64-pc-linux-gnu-gcc -pthread -DNDEBUG -O2 -fPIC -DEXPERIMENTAL=1 -DNDEBUG=1 -DAPSW_FORK_CHECKER=1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -Isrc -I/usr/include/python2.7 -c src/apsw.c -o build/temp.linux-x86_64-2.7/src/apsw.o
In file included from /usr/include/python2.7/Python.h:8:0,
                 from src/apsw.c:73:
/usr/include/python2.7/pyconfig.h:1161:0: warning: "_POSIX_C_SOURCE" redefined [enabled by default]
/usr/include/features.h:215:0: note: this is the location of the previous definition
creating build/lib.linux-x86_64-2.7
x86_64-pc-linux-gnu-gcc -pthread -shared -O2 build/temp.linux-x86_64-2.7/src/apsw.o -L/usr/lib64 -lsqlite3 -lpython2.7 -o build/lib.linux-x86_64-2.7/apsw.so
$ PYTHONPATH="build/lib.linux-x86_64-2.7" python2.7 tests.py -v
                Python /usr/bin/python2.7 sys.version_info(major=2, minor=7, micro=4, releaselevel='alpha', serial=0)
Testing with APSW file /tmp/apsw-3.7.14-r1/build/lib.linux-x86_64-2.7/apsw.so
          APSW version 3.7.14-r1
    SQLite lib version 3.7.14
SQLite headers version 3007014
    Using amalgamation False
testAggregateFunctions (__main__.APSW)
Verify aggregate functions ... Segmentation fault
msg171249 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-09-25 11:17
Can you provide a stacktrace?
msg171254 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-09-25 11:56
I can't build apsw here:

gcc -pthread -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DEXPERIMENTAL=1 -DNDEBUG=1 -DAPSW_FORK_CHECKER=1 -DSQLITE_OMIT_LOAD_EXTENSION=1 -Isrc -I/home/antoine/opt/include/python3.3m -c src/apsw.c -o build/temp.linux-x86_64-3.3/src/apsw.o
src/apsw.c:62:2: erreur: #error Your SQLite version is too old. It must be at least 3.7.13
msg171279 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-09-25 14:13
I don't get a segfault, but a fatal error. Here's the back trace
(unfortunately I've no time to debug this further today; sqlite
is the latest version compiled from source):



#0  0x00007ffff71e6a75 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffff71ea5c0 in *__GI_abort () at abort.c:92
#2  0x00000000005076a0 in Py_FatalError (msg=0x652a88 "PyThreadState_Get: no current thread")
    at Python/pythonrun.c:2360
#3  0x0000000000500a67 in PyThreadState_Get () at Python/pystate.c:423
#4  0x000000000043211f in tupledealloc (op=0x7ffff62ce4c0) at Objects/tupleobject.c:236
#5  0x000000000041fb5a in _Py_Dealloc (op=0x7ffff62ce4c0) at Objects/object.c:1764
#6  0x00000000005d6867 in func_dealloc (op=0x7ffff6b8eea0) at Objects/funcobject.c:564
#7  0x000000000041fb5a in _Py_Dealloc (op=0x7ffff6b8eea0) at Objects/object.c:1764
#8  0x00007ffff68754e2 in FunctionCBInfo_dealloc (self=0x7ffff6317bf0) at src/connection.c:119
#9  0x000000000041fb5a in _Py_Dealloc (op=0x7ffff6317bf0) at Objects/object.c:1764
#10 0x00007ffff687b4f4 in apsw_free_func (funcinfo=0x7ffff6317bf0) at src/connection.c:2221
#11 0x00007ffff65d03f7 in functionDestroy (db=0xd0cb08, p=0x26f4) at sqlite3.c:112342
#12 0x00007ffff65eb7e0 in sqlite3CreateFunc (db=0x8, zFunctionName=<value optimized out>, nArg=-1, enc=1, 
    pUserData=<value optimized out>, xFunc=0, xStep=0x7ffff687ae90 <cbdispatch_step>, 
    xFinal=0x7ffff687b0a4 <cbdispatch_final>, pDestructor=0xd05448) at sqlite3.c:112858
#13 0x00007ffff65ebb9c in sqlite3_create_function_v2 (db=0xd0cb08, zFunc=<value optimized out>, nArg=-1, enc=1, 
    p=0x7ffff6ba3988, xFunc=<value optimized out>, xStep=0x7ffff687ae90 <cbdispatch_step>, 
    xFinal=0x7ffff687b0a4 <cbdispatch_final>, xDestroy=0x7ffff687b47d <apsw_free_func>) at sqlite3.c:112913
#14 0x00007ffff687ba30 in Connection_createaggregatefunction (self=0x7ffff6b2cdd8, args=0x7ffff62f29c0)
    at src/connection.c:2395
#15 0x00000000005ffee7 in PyCFunction_Call (func=0x7ffff6bab768, arg=0x7ffff62f29c0, kw=0x0)
    at Objects/methodobject.c:81
#16 0x00000000004d0ea1 in call_function (pp_stack=0x7fffffff3b78, oparg=2) at Python/ceval.c:4062
#17 0x00000000004cab06 in PyEval_EvalFrameEx (f=0xd0d720, throwflag=0) at Python/ceval.c:2679
#18 0x00000000004cea43 in PyEval_EvalCodeEx (_co=0x7ffff6078940, globals=0x7ffff7eae330, locals=0x0, args=0xd0c598, 
    argcount=1, kws=0xd0c5a0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3433
#19 0x00000000004d14cb in fast_function (func=0x7ffff63597e0, pp_stack=0x7fffffff47e8, n=1, na=1, nk=0)
    at Python/ceval.c:4160
#20 0x00000000004d1074 in call_function (pp_stack=0x7fffffff47e8, oparg=0) at Python/ceval.c:4083
#21 0x00000000004cab06 in PyEval_EvalFrameEx (f=0xd0c3e0, throwflag=0) at Python/ceval.c:2679
#22 0x00000000004cea43 in PyEval_EvalCodeEx (_co=0x7ffff6ae3580, globals=0x7ffff6bf1ba0, locals=0x0, args=0xd0c2e0, 
    argcount=3, kws=0xd0c2f8, kwcount=1, defs=0x7ffff6ad6168, defcount=1, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3433
#23 0x00000000004d14cb in fast_function (func=0x7ffff6c0d420, pp_stack=0x7fffffff5458, n=5, na=3, nk=1)
    at Python/ceval.c:4160
#24 0x00000000004d1074 in call_function (pp_stack=0x7fffffff5458, oparg=258) at Python/ceval.c:4083
#25 0x00000000004cab06 in PyEval_EvalFrameEx (f=0xd0c100, throwflag=0) at Python/ceval.c:2679
#26 0x00000000004cea43 in PyEval_EvalCodeEx (_co=0x7ffff6ae3700, globals=0x7ffff6bf1ba0, locals=0x0, 
---Type <return> to continue, or q <return> to quit---
    args=0x7ffff6368628, argcount=2, kws=0x7ffff7fa9088, kwcount=0, defs=0x7ffff6c14da8, defcount=1, kwdefs=0x0, 
    closure=0x0) at Python/ceval.c:3433
#27 0x00000000005d6e1f in function_call (func=0x7ffff6c0d4e0, arg=0x7ffff6368600, kw=0x7ffff62b8c18)
    at Objects/funcobject.c:633
#28 0x000000000059969b in PyObject_Call (func=0x7ffff6c0d4e0, arg=0x7ffff6368600, kw=0x7ffff62b8c18)
    at Objects/abstract.c:2083
#29 0x00000000004d239b in ext_do_call (func=0x7ffff6c0d4e0, pp_stack=0x7fffffff60f8, flags=3, na=1, nk=0)
    at Python/ceval.c:4377
#30 0x00000000004caf3c in PyEval_EvalFrameEx (f=0x7ffff62ef460, throwflag=0) at Python/ceval.c:2720
#31 0x00000000004cea43 in PyEval_EvalCodeEx (_co=0x7ffff6ae3a00, globals=0x7ffff6bf1ba0, locals=0x0, 
    args=0x7ffff5acb1f0, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3433
#32 0x00000000005d6e1f in function_call (func=0x7ffff6c0d660, arg=0x7ffff5acb1c8, kw=0x0) at Objects/funcobject.c:633
#33 0x000000000059969b in PyObject_Call (func=0x7ffff6c0d660, arg=0x7ffff5acb1c8, kw=0x0) at Objects/abstract.c:2083
#34 0x00000000005b7cc4 in method_call (func=0x7ffff6c0d660, arg=0x7ffff5acb1c8, kw=0x0) at Objects/classobject.c:323
#35 0x000000000059969b in PyObject_Call (func=0x7ffff635c7e0, arg=0x7ffff63257d0, kw=0x0) at Objects/abstract.c:2083
#36 0x00000000004462c2 in slot_tp_call (self=0x7ffff62ce840, args=0x7ffff63257d0, kwds=0x0)
    at Objects/typeobject.c:5349
#37 0x000000000059969b in PyObject_Call (func=0x7ffff62ce840, arg=0x7ffff63257d0, kw=0x0) at Objects/abstract.c:2083
#38 0x00000000004d1d18 in do_call (func=0x7ffff62ce840, pp_stack=0x7fffffff6f48, na=1, nk=0) at Python/ceval.c:4282
#39 0x00000000004d1093 in call_function (pp_stack=0x7fffffff6f48, oparg=1) at Python/ceval.c:4085
#40 0x00000000004cab06 in PyEval_EvalFrameEx (f=0xd0ae60, throwflag=0) at Python/ceval.c:2679
#41 0x00000000004cea43 in PyEval_EvalCodeEx (_co=0x7ffff6c05700, globals=0x7ffff6bbe588, locals=0x0, 
    args=0x7ffff62ed178, argcount=2, kws=0x7ffff7fa9088, kwcount=0, defs=0x7ffff6c306a8, defcount=1, kwdefs=0x0, 
    closure=0x0) at Python/ceval.c:3433
#42 0x00000000005d6e1f in function_call (func=0x7ffff6c291e0, arg=0x7ffff62ed150, kw=0x7ffff635c678)
    at Objects/funcobject.c:633
[...]
msg171280 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012-09-25 14:28
My guess is an extension problem (failing to aqcuire locks?)
msg171283 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-09-25 14:31
The cause for Stefan's traceback looks quite clear:

Connection_createaggregatefunction() release the GIL (through the PYSQLITE_CON_CALL macro) before calling into sqlite3_create_function_v2, which itself calls the destructor for the old aggregate function, aka. apsw_free_func(), which calls Py_DECREF without re-acquiring the GIL.

So apsw needs to sanitize its callbacks implementation here.
msg171286 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-09-25 15:23
Not a blocker then.
msg171315 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-09-25 20:04
[Adding the apsw author to the cc.]


Antoine is right, no Python bug here. I think we can close this.
msg171330 - (view) Author: Roger Binns (rogerbinns) Date: 2012-09-26 07:47
Thanks for finding this problem.  I can repeat it with the patch and am in the process of fixing it.
History
Date User Action Args
2012-09-26 07:47:50rogerbinnssetmessages: + msg171330
2012-09-25 20:04:36skrahsetstatus: pending -> closed

nosy: + rogerbinns
messages: + msg171315

stage: resolved
2012-09-25 15:23:12georg.brandlsetpriority: release blocker -> normal
status: open -> pending
resolution: not a bug
messages: + msg171286
2012-09-25 14:31:53pitrousetmessages: + msg171283
2012-09-25 14:28:20benjamin.petersonsetmessages: + msg171280
2012-09-25 14:13:29skrahsetnosy: + skrah
messages: + msg171279
2012-09-25 11:56:10pitrousetmessages: + msg171254
2012-09-25 11:17:29georg.brandlsetmessages: + msg171249
2012-09-25 11:17:05georg.brandlsetassignee: pitrou
2012-09-25 09:40:43Arfrevercreate