classification
Title: Specialized generic class does not return class attributes in dir
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, kjamieson, miss-islington, xtreak
Priority: normal Keywords: patch

Created on 2021-11-08 19:30 by kjamieson, last changed 2021-12-17 11:33 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29962 merged kj, 2021-12-07 13:57
PR 30166 merged miss-islington, 2021-12-17 09:41
Messages (8)
msg405981 - (view) Author: Kevin Jamieson (kjamieson) Date: 2021-11-08 19:30
This worked in Python 3.6, but in Python 3.7 and later creating a mock with a spec specifying a subscripted generic class does not mock any of the attributes of the class, because those attributes are not returned by dir().

For example:

# cat test.py
from typing import Generic, TypeVar
from unittest import mock

T = TypeVar('T')

class Foo(Generic[T]):
    def bar(self) -> None:
        pass

m = mock.MagicMock(spec=Foo[int])
m.bar()


# python3.11 test.py
Traceback (most recent call last):
  File "/root/test.py", line 11, in <module>
    m.bar()
    ^^^^^^^
  File "/usr/lib/python3.11/unittest/mock.py", line 635, in __getattr__
    raise AttributeError("Mock object has no attribute %r" % name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: Mock object has no attribute 'bar'
msg406267 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-11-13 05:10
This seems to be an issue with typing than mock since mock just uses the output from dir() . I am not able to bisect the relevant change but below is the output of dir(Foo[int]) in Python 3.6 and master.

Python 3.6.9

['__abstractmethods__', '__args__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__extra__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__next_in_mro__', '__orig_bases__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '__tree_hash__', '__weakref__', '_abc_cache', '_abc_generic_negative_cache', '_abc_generic_negative_cache_version', '_abc_registry', '_gorg', 'bar']


master branch : 

['__args__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__instancecheck__', '__le__', '__lt__', '__module__', '__mro_entries__', '__ne__', '__new__', '__or__', '__origin__', '__parameters__', '__reduce__', '__reduce_ex__', '__repr__', '__ror__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasscheck__', '__subclasshook__', '__weakref__', '_inst', '_name', '_paramspec_tvars', '_typevar_types', 'copy_with']
msg406280 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-11-13 15:32
Isn’t the solution to use the unspecialized class?
msg407838 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-12-06 16:45
Not exactly sure if this is a bug, but the reason is that Foo[int] used to be a class, now it's a plain object. It's a change brought in 3.7 by PEP 560.

3.6:
>>> isinstance(Foo[int], type)
True
>>> Foo[int].__dir__
<method '__dir__' of 'object' objects
>>> type(Foo[int].__dir__)
<class 'method_descriptor'>

main:
>>> isinstance(Foo[int], type)
False
>>> Foo[int].__dir__
<built-in method __dir__ of _GenericAlias object at 0x000001B44DF0A850>
>>> type(Foo[int].__dir__)
<class 'method_descriptor'>

The fix is just a 2-line change in either _GenericAlias or _BaseGenericAlias:
+    def __dir__(self):
+        return dir(self.__origin__)

Should we go ahead with this?
msg408705 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-12-16 14:30
New changeset d6e13747161d7b634b47d2d3d212ed3be4a21fab by Ken Jin in branch 'main':
bpo-45755: [typing] Reveal class attributes of generic in generic aliases in `dir()` (GH-29962)
https://github.com/python/cpython/commit/d6e13747161d7b634b47d2d3d212ed3be4a21fab
msg408707 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-12-16 14:35
Hi, I won't be backporting this to 3.9 since I don't like having jarring behavior changes so late into the bugfix cycle of a Python version.

I'm tempted to backport to 3.10. For now, I'll be conservative and just merge it in main. Please do tell me if any of you feel that I should backport to 3.10 too.

3.6-3.8 are in security-fix only mode so they won't get any bugfixes.

Thanks Kevin for the interesting bug report, and Karthikeyan for saving me a ton of time debugging this!
msg408729 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-12-16 19:28
I think a 3.10 backport would be appreciated.
msg408774 - (view) Author: miss-islington (miss-islington) Date: 2021-12-17 11:33
New changeset 87539cc716fab47cd4f501f2441c4ab8e80bce6f by Miss Islington (bot) in branch '3.10':
bpo-45755: [typing] Reveal class attributes of generic in generic aliases in `dir()` (GH-29962)
https://github.com/python/cpython/commit/87539cc716fab47cd4f501f2441c4ab8e80bce6f
History
Date User Action Args
2021-12-17 11:33:23miss-islingtonsetmessages: + msg408774
2021-12-17 09:41:20miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request28383
2021-12-16 19:28:56gvanrossumsetmessages: + msg408729
2021-12-16 14:35:18kjsetstatus: open -> closed
resolution: fixed
messages: + msg408707

stage: patch review -> resolved
2021-12-16 14:30:36kjsetmessages: + msg408705
2021-12-07 13:57:10kjsetkeywords: + patch
stage: patch review
pull_requests: + pull_request28187
2021-12-06 16:45:57kjsetmessages: + msg407838
2021-11-13 15:32:55gvanrossumsetmessages: + msg406280
2021-11-13 05:10:54xtreaksetnosy: + gvanrossum, kj

messages: + msg406267
title: Mock spec with a specialized generic class does not mock class attributes -> Specialized generic class does not return class attributes in dir
2021-11-09 02:50:04xtreaksetnosy: + xtreak
2021-11-08 19:30:49kjamiesoncreate