classification
Title: segfault: PyDict_SetItem: Assertion `value' failed.
Type: crash Stage: resolved
Components: Versions: Python 2.7
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: jcea Nosy List: dmalcolm, flox, jcea
Priority: normal Keywords: patch

Created on 2010-08-24 20:51 by flox, last changed 2010-11-05 00:27 by jcea. This issue is now closed.

Files
File name Uploaded Description Edit
issue9675_cobject.diff flox, 2010-08-25 01:03 Patch, apply to 2.7 review
Messages (9)
msg114823 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-24 20:51
Crash on Python 2.7 branch.

$ ./python -We -c 'import anydbm'
python: Objects/dictobject.c:759: PyDict_SetItem: Assertion `value' failed.
Abandon

It occurs with all optional modules compiled.
msg114824 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-24 20:54
Actually, the issue seems to be in bsddb.

$ ./python -We -c 'import bsddb'
python: Objects/dictobject.c:759: PyDict_SetItem: Assertion `value' failed.
Abandon
msg114836 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-24 21:48
It is probably related to the CObject PendingDeprecationWarning.
For the record, bsddb does not use the new Capsule API in Python 2.7.

ref: http://bugs.python.org/issue7992#msg104140
msg114858 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-08-25 00:47
Importing "bsddb" crashes. Importing "bsddb3" (the updated version I keep independiently of python) DOESN'T crash.

bsddb = 4.8.4

bsddb3 = 5.0.0 (currently).

Checking the changelog, I see this possible cause:

"""
  * Capsule support was buggy. The string passed in to PyCapsule_New() must outlive the capsule.  (Larry Hastings)
"""

It is 3AM in Spain now. Will check when I find some time.
msg114865 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-08-25 01:03
The crash is triggered because PyDict_SetItemString(d, "api", py_api) is called with py_api=NULL when PyCObject_FromVoidPtr returns an error.

A possible workaround is to create a copy of PyCObject_FromVoidPtr (e.g. _PyCObject_FromVoidPtr) which is free of warnings.
This function will be used by the _bsddb module in Python 2.7.

See patch.
msg114869 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-08-25 01:21
The problem is that the "-We" is converting the CObject use to ERRORs, when this API is perfectly legal in Python 2.7.

bsddb 4.8.4 DOES *legally* uses CObjects in Python 2.7.

In pybsddb 5.0.0 I migrated to Capsule, since 5.0 doesn't support python 3.0 anymore (python 3.0 didn't support capsule).

I can commit a "4.8.4.1" version with Capsule support, for I think this crash is overzealot. 

According to http://bugs.python.org/issue7992#msg104140 , CObject is not actually deprecated in Python 2.7, so it should not cause CObjects use with "-We" to be an error. Read the mail thread. In particular: http://mail.python.org/pipermail/python-committers/2010-April/000882.html .

I would suggest to drop DeprecationWarnings for CObjects in 2.7.1.

Comments?.
msg117503 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-09-28 03:16
In the spirit of moving this forward:
http://mail.python.org/pipermail/python-dev/2010-September/104201.html
msg120451 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-11-04 21:41
Move the CObject use to a py3k warning instead of an error in r86178.

I still need to solve improve error management in bsddb. This bug remains open for a while.
msg120462 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-11-05 00:16
Better bsddb error control. Please, review.

"""
[jcea@babylon5 release27-maint]$ ./python
Python 2.7.0+ (release27-maint:86176:86178M, Nov  5 2010, 00:30:) 
[GCC 4.5.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> exit()
"""

"""
[jcea@babylon5 release27-maint]$ ./python -3
Python 2.7.0+ (release27-maint:86176:86178M, Nov  5 2010, 00:30:) 
[GCC 4.5.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
__main__:1: DeprecationWarning: in 3.x, the bsddb module has been removed; please use the pybsddb project instead
/home/python/svn-hg/release27-maint/Lib/bsddb/__init__.py:67: DeprecationWarning: CObject type is not supported in 3.x. Please use capsule objects instead.
  import _bsddb
"""

"""
[jcea@babylon5 release27-maint]$ ./python -3 -Werror
Python 2.7.0+ (release27-maint:86176:86178M, Nov  5 2010, 00:30:)
[GCC 4.5.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import warnings
>>> warnings.filterwarnings('ignore',
... "in 3.x, the bsddb module has been removed; please use the pybsddb project instead",
... category=DeprecationWarning)
>>> import bsddb
Exception DeprecationWarning: DeprecationWarning('CObject type is not supported in 3.x. Please use capsule objects instead.',) in <module '_bsddb' (built-in)> ignored
>>> exit()
"""

Why is my "PyErr_Warn()" not being printed?

The code: (py_api is the CObject, will be NULL if an exception happened)

"""
    if (py_api) {
        PyDict_SetItemString(d, "api", py_api);
        Py_DECREF(py_api);
    } else { /* Something bad happened */
        PyErr_WriteUnraisable(m);
        PyErr_Warn(PyExc_RuntimeWarning,
                "_bsddb/_pybsddb C API will be not available");
        PyErr_Clear();
    }
"""

Commit r86180.

Now I have to up-port this patch to the upcoming pybssddb 5.1.1.
History
Date User Action Args
2010-11-05 00:27:17jceasetstatus: open -> closed
assignee: jcea
resolution: accepted
stage: patch review -> resolved
2010-11-05 00:16:05jceasetmessages: + msg120462
2010-11-04 21:41:24jceasetmessages: + msg120451
2010-09-28 15:46:00dmalcolmsetnosy: + dmalcolm
2010-09-28 03:16:43jceasetmessages: + msg117503
2010-08-25 01:21:50jceasetmessages: + msg114869
2010-08-25 01:03:37floxsetfiles: + issue9675_cobject.diff
keywords: + patch
messages: + msg114865

stage: test needed -> patch review
2010-08-25 00:47:38jceasetmessages: + msg114858
2010-08-24 21:48:38floxsetmessages: + msg114836
2010-08-24 20:55:12floxsetnosy: + jcea
2010-08-24 20:54:40floxsetmessages: + msg114824
2010-08-24 20:51:43floxcreate