classification
Title: Generic Alias attributes nor in dir()
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: AllanDaemon, BTaskaya, docs@python, gvanrossum, miss-islington
Priority: normal Keywords: patch

Created on 2020-09-14 07:06 by AllanDaemon, last changed 2020-09-15 23:38 by gvanrossum. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22262 merged BTaskaya, 2020-09-15 20:33
PR 22264 merged miss-islington, 2020-09-15 21:58
Messages (8)
msg376873 - (view) Author: Allan Daemon (AllanDaemon) Date: 2020-09-14 07:06
The implementation of PEP 585 in 3.9 adds some new attributes, but they aren't listed with dir() (then, not list in autocomplete and IntelliSense).

Python 3.9.0rc1+ (heads/3.9:d7cd1164c1, Aug 25 2020, 17:27:09)

>>> li = list[int]
>>> li.__origin__
<class 'list'>
>>> getattr(li, '__origin__')
<class 'list'>
>>> '__origin__' in dir(li)
False

That applies to:

__origin__
__args__
__parameters__

It seems to be not the expected behaviour, so this bug report. If this is not the case, some piece of documentation could be interesting. Also, I couldn't find anything about Generic Alias in the documentation, only in the source code. Should this be addressed too?
msg376897 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-14 19:42
I actually have no idea where dir() gets its information.  IIRC the implementation was a collaboration with Batuhan -- @Batuhan, do you want to look into this?  Presumably some metadata needs to be added to genericaliasobject.c.
msg376898 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-09-14 19:43
> @Batuhan, do you want to look into this?

Sure, will check it out!
msg376936 - (view) Author: Allan Daemon (AllanDaemon) Date: 2020-09-15 11:35
> I actually have no idea where dir() gets its information.

I will share what I find out already, so it may hopefully be helpful.

The dir function used is the general object.__dir__ function, defined in Objects/typeobject.c:type___dir___impl(), that just takes everything from __dict__ of the class and its bases.

But the GenericAlias is trying hard pretending to be the origin class, so when you ask for the __dict__, it gives the original class, not the GenericAlias:

>>> list[int].__dict__ == list.__dict__
True

There are also some other things that seems a bit strange, but it may just be right:

>>> list.__class__
<class 'type'>
>>> list[int].__class__
<class 'type'>

>>> type(list)
<class 'type'>
>>> type(list[int])
<class 'types.GenericAlias'>

This happens because of the tp_getattro function genericaliasobject.c:ga_getattro(), that takes the attributes from the origin except for the following names, that are taken from the GenericAlias itself:
    "__origin__",
    "__args__",
    "__parameters__",
    "__mro_entries__",
    "__reduce_ex__",
    "__reduce__",

So, when the dir() is getting the __dict__, it gets list.__dict__ and base (object) instead of GenericAlias. Looking the ga_getattro() calls when dir() is fired, it also gets the __class__ attribute after getting the __dict__. Adding one or another makes the dir() showing up the GenericAlias attributes, but others went missing.

An idea to solve this spefic issue could be overriding the __dir__ method, 
something like this python code (written in c, of course):

def __dir__(self):
    r = super().__dir__()
    return r + attribute_names_list + method_names_list


But I wonder what else should not be forwarded to origin.
msg376940 - (view) Author: Batuhan Taskaya (BTaskaya) * (Python committer) Date: 2020-09-15 14:19
Okay, so IIRC if we stop forwarding __class__ (add it as an exception to attr_exceptions) it would return us the original dir(), and also solve the inconsistency of the example you gave;

> >>> list.__class__
> <class 'type'>
> >>> list[int].__class__
> <class 'type'>
>
> >>> type(list)
> <class 'type'>
> >>> type(list[int])
> <class 'types.GenericAlias'>

but the problem is that, it is not 'exactly' complying with the specs at PEP 585. I am aware that we already added some extras to this list (https://www.python.org/dev/peps/pep-0585/#parameters-to-generics-are-available-at-runtime) like __mro_entries__ etc, but __class__ looks like a tricky problem. @gvanrossum any comments?
msg376948 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-09-15 17:51
The forwarding of `__class__`, and the resulting difference between `type(list[int])` and `list[int].__class__` are intentional.

I think overriding `__dir__` along the lines suggested by Allan (but in C of course) would be the best solution.
msg376957 - (view) Author: miss-islington (miss-islington) Date: 2020-09-15 21:58
New changeset 2e87774df1a0eaf2a1fe8cc4d958df60f7125b6e by Batuhan Taskaya in branch 'master':
bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262)
https://github.com/python/cpython/commit/2e87774df1a0eaf2a1fe8cc4d958df60f7125b6e
msg376965 - (view) Author: miss-islington (miss-islington) Date: 2020-09-15 23:37
New changeset 49917d576a578c8c29b8dbcbd5257c366a53d498 by Miss Islington (bot) in branch '3.9':
bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262)
https://github.com/python/cpython/commit/49917d576a578c8c29b8dbcbd5257c366a53d498
History
Date User Action Args
2020-09-15 23:38:50gvanrossumsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-09-15 23:37:21miss-islingtonsetmessages: + msg376965
2020-09-15 21:58:50miss-islingtonsetpull_requests: + pull_request21320
2020-09-15 21:58:40miss-islingtonsetnosy: + miss-islington
messages: + msg376957
2020-09-15 20:33:44BTaskayasetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request21318
2020-09-15 17:51:41gvanrossumsetassignee: docs@python ->
stage: needs patch
2020-09-15 17:51:16gvanrossumsetmessages: + msg376948
2020-09-15 14:19:52BTaskayasetmessages: + msg376940
2020-09-15 11:35:58AllanDaemonsetmessages: + msg376936
components: + Interpreter Core, - Documentation, Library (Lib)
2020-09-14 19:43:46BTaskayasetmessages: + msg376898
2020-09-14 19:42:45gvanrossumsetnosy: + BTaskaya
messages: + msg376897
2020-09-14 08:01:22xtreaksetnosy: + gvanrossum
2020-09-14 07:06:06AllanDaemoncreate