classification
Title: Support key-sharing dictionaries in subclasses
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, Trundle, benjamin.peterson, haypo, larry, pingebretson, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2014-02-15 22:49 by pingebretson, last changed 2014-03-17 21:00 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
subclass-keys-pep-412.patch pingebretson, 2014-02-15 22:49 Patch to support key-sharing dictionaries in subclasses
subclass-keys-pep-412.patch pingebretson, 2014-02-16 20:22 Revised patch based on review feedback review
Messages (10)
msg211302 - (view) Author: Peter Ingebretson (pingebretson) * Date: 2014-02-15 22:49
PEP 412 shared keys are not created for subclasses in Python 3.3 and 3.4:

>>> import sys
>>> class A:
...     pass
... 
>>> class B(A):
...     pass
... 
>>> a, b = A(), B()
>>> sys.getsizeof(vars(a))
96
>>> sys.getsizeof(vars(b))
288

(Actual sizes depend on platform and configuration).

This patch allows subclasses to share keys:

>>> import sys
>>> class A:
...     pass
... 
>>> class B(A):
...     pass
... 
>>> a, b = A(), B()
>>> sys.getsizeof(vars(a))
96
>>> sys.getsizeof(vars(b))
96
msg211315 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-16 13:16
Wow, really? Thanks for finding this!
msg211320 - (view) Author: Mark Shannon (Mark.Shannon) * Date: 2014-02-16 15:27
Well spotted.

I don't think that the line 
COPYVAL(tp_dictoffset);
should be removed.

inherit_special() is called from PyType_Ready
which is used to initialise static TypeObjects.
So builtin class B which doesn't set tp_dictoffset,
but has builtin class A as its base, will not get its
tp_dictoffset initialized from A.
msg211358 - (view) Author: Peter Ingebretson (pingebretson) * Date: 2014-02-16 20:22
Thanks Mark, I agree.

I thought that assigning to tp_dictoffset in inherit_special was redundant with the assignment in inherit_slot and the assignment I added, but I see that COPYSLOT and COPYVAL are slightly different and extension types won't go through type_new.

Here's an updated patch.
msg211376 - (view) Author: Mark Shannon (Mark.Shannon) * Date: 2014-02-16 23:39
This looks good to me.

Also, I think this should go into 3.4
It is a tiny patch and could result in significant memory saving for OO programs with inheritance (e.g. Django)
msg211378 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-02-16 23:46
> Also, I think this should go into 3.4

I think that it's too late for optimizations in Python 3.4 (RC1 is alread out). It can wait Python 3.5, or maybe 3.4.1.
msg211385 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-17 02:17
I agree: definitely not for 3.5, maybe for 3.4.1.
msg212002 - (view) Author: Roundup Robot (python-dev) Date: 2014-02-23 15:51
New changeset 16229573e73e by Antoine Pitrou in branch 'default':
Issue #20637: Key-sharing now also works for instance dictionaries of subclasses.  Patch by Peter Ingebretson.
http://hg.python.org/cpython/rev/16229573e73e
msg212003 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-02-23 15:52
Thanks. I've committed a patch after augmenting the tests a bit, so it'll be in 3.4.1 as well as 3.5.
msg213895 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-17 21:00
New changeset afae24cb81d9 by Benjamin Peterson in branch '3.4':
correct the fix for #20637; allow slot descriptor inheritance to take place before creating cached keys
http://hg.python.org/cpython/rev/afae24cb81d9
History
Date User Action Args
2014-03-17 21:00:36python-devsetmessages: + msg213895
2014-02-23 15:52:07pitrousetstatus: open -> closed
versions: + Python 3.4
messages: + msg212003

resolution: fixed
stage: patch review -> resolved
2014-02-23 15:51:16python-devsetnosy: + python-dev
messages: + msg212002
2014-02-17 02:17:39larrysetnosy: + larry
messages: + msg211385
2014-02-16 23:46:57hayposetnosy: + haypo
messages: + msg211378
2014-02-16 23:39:50Mark.Shannonsetmessages: + msg211376
2014-02-16 21:35:58Trundlesetnosy: + Trundle
2014-02-16 20:22:20pingebretsonsetfiles: + subclass-keys-pep-412.patch

messages: + msg211358
2014-02-16 15:27:26Mark.Shannonsetmessages: + msg211320
2014-02-16 13:16:23pitrousetnosy: + Mark.Shannon, pitrou, benjamin.peterson

messages: + msg211315
stage: patch review
2014-02-15 22:49:04pingebretsoncreate