This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: replacing obj.__dict__ with a subclass of dict
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: ajaksu2, eric.snow, gangesmaster, iritkatriel, josh.r, kushal.das, matthieu.labbe, r.david.murray, rhettinger, serhiy.storchaka, torsten
Priority: low Keywords:

Created on 2006-04-24 17:45 by gangesmaster, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (10)
msg60907 - (view) Author: ganges master (gangesmaster) Date: 2006-04-24 17:45
>>> class mydict(dict):
...     def __getitem__(self, key):
...         return 17
...
>>> class blah(object):
...     def __init__(self):
...         self.__dict__ = mydict()
...
>>> b = blah()
>>> print b.x
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'blah' object has no attribute 'x'

python doesn't call the overriden version of __getitem__.  

i've done several more tests, and the cause to this
problem, afaik, is that the code assumes __dict__ is an
instance of dict, so it directly uses PyDict_GetItem
(or whatever it's called), thus skipping the overriden
method.

python should either disable setting __dict__ to
anything that is not a real dict (type(x) == dict
instead of isinstance(x, dict)), or be willing to call
overriden methods.
msg83910 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-03-21 02:20
Confirmed, is this a valid issue?
msg92419 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-09-08 15:25
2.5 and 2.4 are in security-fix-only mode, so we don't set them in
versions since bugs won't get fixed there.

I don't think overridden methods should be called, since that would slow
down attribute lookup, which is already a bottleneck in Python.  Whether
there should be an error message is a more complicated question, and I'd
have to look at how this is implemented to even have an opinion on that.
msg188082 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2013-04-29 18:26
In Objects/typeobject.c we have subtype_setdict function, in which at line 1830 we have PyDict_Check() macro call, which checks if it is a subclass of dict or not.

The definition is in Include/dictobject.h

#define PyDict_Check(op) \
                 PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_DICT_SUBCLASS)

We can stop assigning anything other than dict in typesobject.c but that will break other things like http://bugs.python.org/issue1475692
msg259213 - (view) Author: Torsten Landschoff (torsten) * Date: 2016-01-29 16:16
I just bumped into this issue because I was shown by a colleague that my implementation of immutable objects (by replacing __dict__ with an ImmutableDict that inherits from dict and blocks write accesses) is ineffective - ouch!

I'd expect that Python either rejects subclasses of dict for obj.__dict__ or actually implements accessing correctly. Especially since the enum module created the impression for me that replacing __dict__ is a viable approach (enum.py uses __prepare__ in the meta class to provide a different dict class for enum types, just found https://www.python.org/dev/peps/pep-3115/).

Interestingly the PEP 3115 example code notes the following:

# Note that we replace the classdict with a regular
# dict before passing it to the superclass, so that we
# don't continue to record member names after the class
# has been created.
msg259215 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-29 17:13
Python uses concrete class API (PyDict_GetItem and like) for resolving attributes. Using general mapping API would slow down attribute lookup in common case. This is performance critical part of Python and we should be very careful changing it.

On the other side, you still can have a benefit from using dict subclasses as __dict__. OrderedDict allows you to iterate dict in predefined order, and defaultdict allows you to create attributes with default values on demand (but __getattr__ is more universal method).

See also issue10977.
msg259257 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2016-01-30 15:23
Just FYI, if you're only trying to make immutable objects, that's what subclassing tuple with properties and __slots__ = () is for (collections.namedtuple does exactly this, building the Python declaration as a string and then eval-ing it to produce a tuple subclass with named property accessors). The only negative is that it still acts like a sequence, but usually that's not a big problem.
msg380486 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-11-07 01:40
Calling the overridden __getitem__ is rejected due to performance.

Forbidding dict subclasses is rejected because subclasses like ordereddict and defaultdict can be useful.

I think the only remaining possibilities are to do nothing or to raise an error when __dict__ is set to a dict subclass that overrides __getitem__ (that should be ok to do, because that __getitem__ is not going to be called).
msg380523 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-11-07 20:13
I'm going to mark this as rejected.  After 14 years, no clean and performant solution has emerged, yet the world of Python seems to be fine with the status quo.  This issue doesn't seem to be getting in the way of people doing their job.
msg380525 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-07 21:56
There is no longer need to use OrderedDict as __dict__, but ctypes types have tp_dict which are instances of dict subclass (StgDict). Disabling setting it to anything that is not an exact dict would break ctypes.
History
Date User Action Args
2022-04-11 14:56:17adminsetgithub: 43272
2020-11-07 21:56:36serhiy.storchakasetmessages: + msg380525
2020-11-07 20:13:38rhettingersetstatus: open -> closed
resolution: wont fix
messages: + msg380523

stage: test needed -> resolved
2020-11-07 01:40:42iritkatrielsetnosy: + iritkatriel

messages: + msg380486
versions: + Python 3.8, Python 3.9, Python 3.10, - Python 3.1, Python 2.7, Python 3.2
2016-01-30 15:23:32josh.rsetnosy: + josh.r
messages: + msg259257
2016-01-29 17:13:35serhiy.storchakasetnosy: + serhiy.storchaka, rhettinger
messages: + msg259215
2016-01-29 16:16:30torstensetnosy: + torsten
messages: + msg259213
2014-05-12 17:59:55eric.snowsetnosy: + eric.snow
2013-04-29 18:26:23kushal.dassetnosy: + kushal.das
messages: + msg188082
2010-08-22 09:41:32BreamoreBoysetversions: - Python 2.6
2009-09-08 15:25:50r.david.murraysetpriority: normal -> low
versions: + Python 2.6, Python 3.2, - Python 2.5, Python 2.4
nosy: + r.david.murray

messages: + msg92419
2009-09-08 14:59:03matthieu.labbesettype: enhancement -> behavior
2009-09-08 14:57:22matthieu.labbesetnosy: + matthieu.labbe

versions: + Python 2.5, Python 2.4
2009-03-21 02:20:55ajaksu2setversions: + Python 3.1, Python 2.7, - Python 2.4
nosy: + ajaksu2

messages: + msg83910

type: enhancement
stage: test needed
2006-04-24 17:45:28gangesmastercreate