Issue40098
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-03-28 19:47 by serhiy.storchaka, last changed 2022-04-11 14:59 by admin.
Messages (11) | |||
---|---|---|---|
msg365227 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2020-03-28 19:47 | |
Due to the difference in the code of __getattr__ and __dir__ for object and type dir() does not return the list of valid attributes for the object. It can return a name which is not a valid attribute and miss a name of the valid attribute. 1. It does not support metaclasses. class M(type): x = 1 class A(metaclass=M): pass assert hasattr(A, 'x') assert 'x' not in dir(A) 2. It does not use __mro__, but uses __bases__ recursively. class M(type): def mro(cls): if cls.__name__ == 'A': return cls, B return cls, class B(metaclass=M): x = 1 class A(metaclass=M): pass assert hasattr(A, 'x') assert 'x' not in dir(A) 3. It uses the __dict__ attribute instead of the instance dict (they can be different). class A: @property def __dict__(self): return {'x': 1} assert not hasattr(A(), 'x') assert 'x' in dir(A()) 4. It uses the __class__ attribute instead of type(). class B: y = 2 class A: x = 1 @property def __class__(self): return B assert hasattr(A, 'x') assert not hasattr(A, 'y') assert hasattr(A(), 'x') assert not hasattr(A(), 'y') assert 'x' in dir(A) assert 'y' not in dir(A) assert 'x' not in dir(A()) assert 'y' in dir(A()) 4.1. As a side effect dir() creates an instance dictionary if it was not initialized yet (for memory saving). It is possible to make these implementations of __dir__() returning exactly what the corresponding __getattr__() accepts, not more and not less. The code will even be much simpler. But is it what we want? |
|||
msg365271 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-29 18:02 | |
I think you're going too far for some of these. > 1. metaclasses This is reasonable. > 2. __mro__ This is also reasonable. (I wonder if that part of the dir() implementation predates __mro__?) I'm not sure about honoring mro() in the metaclass, but it may be useful. > 3. __dict__ I think this is probably a feature -- I can't think of a reason to override __dict__ as a property except when implementing some sort of proxy class, and then it's likely that there's a matching overrid of __getattr__. > 4. __class__ Again, I think this is a feature. For example, PEP 585 overrides __class__: >>> t = list[int] >>> type(t) <class 'types.GenericAlias'> >>> t.__class__ <class 'type'> > 4.1 auto-creation of instance dict This seems an accident of implementation, and if you can avoid it, that's better. (Though what use case of dir() currently suffers from memory overhead due to this?) |
|||
msg365272 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-29 18:06 | |
FWIW it might be a good idea to look into how PEP 585 could benefit from the improvements to dir(). Currently, dir(list) and dir(list[int]) are quite different -- only the former shows list methods like append and insert. See https://github.com/gvanrossum/cpython/tree/pep585 |
|||
msg365411 - (view) | Author: Vedran Čačić (veky) * | Date: 2020-03-31 19:35 | |
Guido wrote: >>> t = list[int] A few years ago, I tried to explain to you that it's much more intuitive to use builtin types instead of their alter egos from typing (List etc.) for [] operator, you said it's not important. It's funny that you've implicitly acknowledged what I said through this typo. ;-) |
|||
msg365416 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-31 20:04 | |
> >>> t = list[int] > > A few years ago, I tried to explain to you that it's much more intuitive > to use builtin types instead of their alter egos from typing (List etc.) > for [] operator, you said it's not important. It's funny that you've > implicitly acknowledged what I said through this typo. ;-) It wasn't a typo. See PEPE 585. Its time for this change *now*. It wasn't *then*. |
|||
msg365417 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2020-03-31 20:19 | |
5. dir(module) does not contain module type attributes (in contrary to dir() for regular object). >>> import keyword >>> dir(keyword) ['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'iskeyword', 'kwlist'] >>> sorted(object.__dir__(keyword)) ['__all__', '__builtins__', '__cached__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__file__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__loader__', '__lt__', '__name__', '__ne__', '__new__', '__package__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__spec__', '__str__', '__subclasshook__', 'iskeyword', 'kwlist'] Seems dir() was initially designed for old-style objects and many changes in object model were passed its. On one hand, it would look logical if dir() returned the list of all names that can be used as attributes. But my concerns are that it may add a "noise", names which can be legally used as attribute names, but which are rarely accessed as instance attribute instead of type attributes. This applies to 1 (metaclasses) and 5 (module type). If left out 1 and 5 and implement all other options (and I remind that it simplifies the code), it will only break one test for unittest.mock. I'm exploring what's going on there. |
|||
msg365419 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-31 20:24 | |
For me, one of the most annoying things about dir() is that it gives all the dunders. There are many dunders that are just always there (__class__, mostly __dict__, __doc__, __name__ etc.). I wish it would just not give dunders that are inherited from object or type. Though it's probably too late for that (people's code might break). |
|||
msg365420 - (view) | Author: Vedran Čačić (veky) * | Date: 2020-03-31 20:33 | |
Guido: First, wow! Second, now it's even more glaring that list[str] has repr 'list[str]', while list doesn't have a repr 'list'. :-( Is it possible that the time for _that_ change will also come some day? :-) Third, my problem with dir is not dunders, but dunders that really shouldn't be there. For example, __le__ when type doesn't support ordering. I understand how that happens, but it doesn't mean it isn't a bug. |
|||
msg365423 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-31 20:40 | |
Vedran, please stay on topic for this issue. FWIW I agree that it would be best if dir() showed only those dunders that are significant. E.g. __eq__ should only be shown if it is overridden by a subclass. Serhiy did you see my feedback on 3 and 4? |
|||
msg365426 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * | Date: 2020-03-31 21:05 | |
If using __class__ and __dict__ attribuites is a feature, it does not work for __getattr__(). I think it is confusing if __dir__() and __getattr__() are not consistent. I'll try to deal with unittes.mock and then will look how this will work with other classes that override __class__ or __dict__. Also we need to check all classes with custom __getattr__ for the matter of custom __dir__. I think that it might be better if dir() for instance excluded names which are not accessed as via instance, e.g. non-descriptor class attributes and class methods. But it may be not easy to do. |
|||
msg365428 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2020-03-31 21:33 | |
At least for overriding __dict__, I would presume there would be a matching override for __getattribute__ (not __getattr__). Ditto for __class__, e.g. look at GenericAlias in the pep585 implementation PR. (Here __class__ is not in the list of exceptions so it returns the attribute from list.) |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:28 | admin | set | github: 84279 |
2020-03-31 21:33:43 | gvanrossum | set | messages: + msg365428 |
2020-03-31 21:05:18 | serhiy.storchaka | set | messages: + msg365426 |
2020-03-31 20:40:26 | gvanrossum | set | messages: + msg365423 |
2020-03-31 20:33:08 | veky | set | messages: + msg365420 |
2020-03-31 20:24:42 | gvanrossum | set | messages: + msg365419 |
2020-03-31 20:19:01 | serhiy.storchaka | set | messages: + msg365417 |
2020-03-31 20:04:10 | gvanrossum | set | messages: + msg365416 |
2020-03-31 19:35:51 | veky | set | nosy:
+ veky messages: + msg365411 |
2020-03-29 18:06:38 | gvanrossum | set | messages: + msg365272 |
2020-03-29 18:02:58 | gvanrossum | set | messages: + msg365271 |
2020-03-28 19:47:28 | serhiy.storchaka | create |