classification
Title: replacing obj.__dict__ with a subclass of dict
Type: behavior Stage: test needed
Components: Interpreter Core Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ajaksu2, eric.snow, gangesmaster, josh.r, kushal.das, matthieu.labbe, r.david.murray, rhettinger, serhiy.storchaka, torsten
Priority: low Keywords:

Created on 2006-04-24 17:45 by gangesmaster, last changed 2016-01-30 15:23 by josh.r.

Messages (7)
msg60907 - (view) Author: ganges master (gangesmaster) Date: 2006-04-24 17:45
>>> class mydict(dict):
...     def __getitem__(self, key):
...         return 17
...
>>> class blah(object):
...     def __init__(self):
...         self.__dict__ = mydict()
...
>>> b = blah()
>>> print b.x
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'blah' object has no attribute 'x'

python doesn't call the overriden version of __getitem__.  

i've done several more tests, and the cause to this
problem, afaik, is that the code assumes __dict__ is an
instance of dict, so it directly uses PyDict_GetItem
(or whatever it's called), thus skipping the overriden
method.

python should either disable setting __dict__ to
anything that is not a real dict (type(x) == dict
instead of isinstance(x, dict)), or be willing to call
overriden methods.
msg83910 - (view) Author: Daniel Diniz (ajaksu2) Date: 2009-03-21 02:20
Confirmed, is this a valid issue?
msg92419 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-09-08 15:25
2.5 and 2.4 are in security-fix-only mode, so we don't set them in
versions since bugs won't get fixed there.

I don't think overridden methods should be called, since that would slow
down attribute lookup, which is already a bottleneck in Python.  Whether
there should be an error message is a more complicated question, and I'd
have to look at how this is implemented to even have an opinion on that.
msg188082 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2013-04-29 18:26
In Objects/typeobject.c we have subtype_setdict function, in which at line 1830 we have PyDict_Check() macro call, which checks if it is a subclass of dict or not.

The definition is in Include/dictobject.h

#define PyDict_Check(op) \
                 PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_DICT_SUBCLASS)

We can stop assigning anything other than dict in typesobject.c but that will break other things like http://bugs.python.org/issue1475692
msg259213 - (view) Author: Torsten Landschoff (torsten) * Date: 2016-01-29 16:16
I just bumped into this issue because I was shown by a colleague that my implementation of immutable objects (by replacing __dict__ with an ImmutableDict that inherits from dict and blocks write accesses) is ineffective - ouch!

I'd expect that Python either rejects subclasses of dict for obj.__dict__ or actually implements accessing correctly. Especially since the enum module created the impression for me that replacing __dict__ is a viable approach (enum.py uses __prepare__ in the meta class to provide a different dict class for enum types, just found https://www.python.org/dev/peps/pep-3115/).

Interestingly the PEP 3115 example code notes the following:

# Note that we replace the classdict with a regular
# dict before passing it to the superclass, so that we
# don't continue to record member names after the class
# has been created.
msg259215 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-29 17:13
Python uses concrete class API (PyDict_GetItem and like) for resolving attributes. Using general mapping API would slow down attribute lookup in common case. This is performance critical part of Python and we should be very careful changing it.

On the other side, you still can have a benefit from using dict subclasses as __dict__. OrderedDict allows you to iterate dict in predefined order, and defaultdict allows you to create attributes with default values on demand (but __getattr__ is more universal method).

See also issue10977.
msg259257 - (view) Author: Josh Rosenberg (josh.r) * Date: 2016-01-30 15:23
Just FYI, if you're only trying to make immutable objects, that's what subclassing tuple with properties and __slots__ = () is for (collections.namedtuple does exactly this, building the Python declaration as a string and then eval-ing it to produce a tuple subclass with named property accessors). The only negative is that it still acts like a sequence, but usually that's not a big problem.
History
Date User Action Args
2016-01-30 15:23:32josh.rsetnosy: + josh.r
messages: + msg259257
2016-01-29 17:13:35serhiy.storchakasetnosy: + serhiy.storchaka, rhettinger
messages: + msg259215
2016-01-29 16:16:30torstensetnosy: + torsten
messages: + msg259213
2014-05-12 17:59:55eric.snowsetnosy: + eric.snow
2013-04-29 18:26:23kushal.dassetnosy: + kushal.das
messages: + msg188082
2010-08-22 09:41:32BreamoreBoysetversions: - Python 2.6
2009-09-08 15:25:50r.david.murraysetpriority: normal -> low
versions: + Python 2.6, Python 3.2, - Python 2.5, Python 2.4
nosy: + r.david.murray

messages: + msg92419
2009-09-08 14:59:03matthieu.labbesettype: enhancement -> behavior
2009-09-08 14:57:22matthieu.labbesetnosy: + matthieu.labbe

versions: + Python 2.5, Python 2.4
2009-03-21 02:20:55ajaksu2setversions: + Python 3.1, Python 2.7, - Python 2.4
nosy: + ajaksu2

messages: + msg83910

type: enhancement
stage: test needed
2006-04-24 17:45:28gangesmastercreate