Author xiang.zhang
Recipients inada.naoki, jaraco, ned.deily, python-dev, rhettinger, serhiy.storchaka, vstinner, xiang.zhang
Date 2016-11-01.16:09:34
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1478016576.12.0.942115372029.issue28199@psf.upfronthosting.co.za>
In-reply-to
Content
I use gdb to run setuptools test suite and find the assumption, split tables are always dense is broken for both dictresize3 and dictresize4.

#0  0x00007ffff71171c7 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007ffff7118e2a in __GI_abort () at abort.c:89
#2  0x00007ffff71100bd in __assert_fail_base (fmt=0x7ffff7271f78 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x5e4b90 "oldvalues[i] != ((void *)0)", file=file@entry=0x5e4aa0 "Objects/dictobject.c", line=line@entry=1270, 
    function=function@entry=0x5e59f0 <__PRETTY_FUNCTION__.12083> "dictresize") at assert.c:92
#3  0x00007ffff7110172 in __GI___assert_fail (assertion=assertion@entry=0x5e4b90 "oldvalues[i] != ((void *)0)", 
    file=file@entry=0x5e4aa0 "Objects/dictobject.c", line=line@entry=1270, function=function@entry=0x5e59f0 <__PRETTY_FUNCTION__.12083> "dictresize")
    at assert.c:101
#4  0x000000000048bddc in dictresize (mp=mp@entry=0x7ffff219d2b0, minused=<optimized out>) at Objects/dictobject.c:1270
#5  0x000000000048bf93 in insertion_resize (mp=mp@entry=0x7ffff219d2b0) at Objects/dictobject.c:1100
#6  0x000000000048c5fd in insertdict (mp=mp@entry=0x7ffff219d2b0, key=key@entry=0x7ffff579c3c0, hash=-3681610201421769281, 
    value=value@entry=0x7ffff07f56e8) at Objects/dictobject.c:1136
#7  0x000000000048fdfd in PyDict_SetItem (op=op@entry=0x7ffff219d2b0, key=key@entry=0x7ffff579c3c0, value=value@entry=0x7ffff07f56e8)
    at Objects/dictobject.c:1572
#8  0x0000000000492cb5 in _PyObjectDict_SetItem (tp=tp@entry=0xd52548, dictptr=0x7ffff080cbd8, key=key@entry=0x7ffff579c3c0, 
    value=value@entry=0x7ffff07f56e8) at Objects/dictobject.c:4274
#9  0x000000000049df8a in _PyObject_GenericSetAttrWithDict (obj=0x7ffff080cbb8, name=0x7ffff579c3c0, value=0x7ffff07f56e8, dict=dict@entry=0x0)
    at Objects/object.c:1172
#10 0x000000000049e0cf in PyObject_GenericSetAttr (obj=<optimized out>, name=<optimized out>, value=<optimized out>) at Objects/object.c:1194
#11 0x000000000049d80e in PyObject_SetAttr (v=v@entry=0x7ffff080cbb8, name=name@entry=0x7ffff579c3c0, value=value@entry=0x7ffff07f56e8)
    at Objects/object.c:932

Thanks to Victor's _PyDict_CheckConsistency, it's easy then to find even without dictresize3 and dictresize4 (the current version), the test suite still fails (#define DEBUG_PYDICT).

#0  0x00007ffff71171c7 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007ffff7118e2a in __GI_abort () at abort.c:89
#2  0x00007ffff71100bd in __assert_fail_base (fmt=0x7ffff7271f78 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x5e53a0 "mp->ma_values[i] != ((void *)0)", file=file@entry=0x5e4d00 "Objects/dictobject.c", line=line@entry=498, 
    function=function@entry=0x5e5dd0 <__PRETTY_FUNCTION__.11869> "_PyDict_CheckConsistency") at assert.c:92
#3  0x00007ffff7110172 in __GI___assert_fail (assertion=assertion@entry=0x5e53a0 "mp->ma_values[i] != ((void *)0)", 
    file=file@entry=0x5e4d00 "Objects/dictobject.c", line=line@entry=498, 
    function=function@entry=0x5e5dd0 <__PRETTY_FUNCTION__.11869> "_PyDict_CheckConsistency") at assert.c:101
#4  0x000000000048ba17 in _PyDict_CheckConsistency (mp=mp@entry=0x7ffff0806e68) at Objects/dictobject.c:498
#5  0x00000000004927a3 in PyDict_SetDefault (d=d@entry=0x7ffff0806e68, key=0x7ffff2ffcdd8, defaultobj=0x8abf20 <_Py_NoneStruct>)
    at Objects/dictobject.c:2807
#6  0x0000000000492854 in dict_setdefault (mp=0x7ffff0806e68, args=<optimized out>) at Objects/dictobject.c:2824
#7  0x0000000000499469 in _PyCFunction_FastCallDict (func_obj=func_obj@entry=0x7ffff0f2f8c8, args=args@entry=0x105afe8, nargs=nargs@entry=2, 
    kwargs=kwargs@entry=0x0) at Objects/methodobject.c:234
#8  0x0000000000499815 in _PyCFunction_FastCallKeywords (func=func@entry=0x7ffff0f2f8c8, stack=stack@entry=0x105afe8, nargs=nargs@entry=2, 
    kwnames=kwnames@entry=0x0) at Objects/methodobject.c:295
#9  0x0000000000537b6f in call_function (pp_stack=pp_stack@entry=0x7fffffff5cd0, oparg=oparg@entry=2, kwnames=kwnames@entry=0x0)
    at Python/ceval.c:4793

From the backtrace we can see PyDict_SetDefault breaks the invariant. And reading the code, yes, it doesn't handle split table separately.

I simply replace the logic in PyDict_SetDefault with insertdict to make a test. It doesn't fail, even with dictresize4.

An easy example to reproduce:

>>> class C:
...     pass
... 
>>> c1, c2 = C(), C()
>>> c1.a, c1.b = 1, 2
>>> c2.__dict__.setdefault('b', None)
python: Objects/dictobject.c:498: _PyDict_CheckConsistency: Assertion `mp->ma_values[i] != ((void *)0)' failed.
Aborted (core dumped)
History
Date User Action Args
2016-11-01 16:09:36xiang.zhangsetrecipients: + xiang.zhang, rhettinger, jaraco, vstinner, ned.deily, inada.naoki, python-dev, serhiy.storchaka
2016-11-01 16:09:36xiang.zhangsetmessageid: <1478016576.12.0.942115372029.issue28199@psf.upfronthosting.co.za>
2016-11-01 16:09:36xiang.zhanglinkissue28199 messages
2016-11-01 16:09:34xiang.zhangcreate