Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Alias attributes nor in dir() #85946

Closed
AllanDaemon mannequin opened this issue Sep 14, 2020 · 8 comments
Closed

Generic Alias attributes nor in dir() #85946

AllanDaemon mannequin opened this issue Sep 14, 2020 · 8 comments
Labels
3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@AllanDaemon
Copy link
Mannequin

AllanDaemon mannequin commented Sep 14, 2020

BPO 41780
Nosy @gvanrossum, @miss-islington, @isidentical, @AllanDaemon
PRs
  • bpo-41780: Fix __dir__ of types.GenericAlias #22262
  • [3.9] bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262) #22264
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-09-15.23:38:50.719>
    created_at = <Date 2020-09-14.07:06:06.931>
    labels = ['interpreter-core', 'type-bug', '3.9', '3.10']
    title = 'Generic Alias attributes nor in dir()'
    updated_at = <Date 2020-09-15.23:38:50.719>
    user = 'https://github.com/AllanDaemon'

    bugs.python.org fields:

    activity = <Date 2020-09-15.23:38:50.719>
    actor = 'gvanrossum'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-09-15.23:38:50.719>
    closer = 'gvanrossum'
    components = ['Interpreter Core']
    creation = <Date 2020-09-14.07:06:06.931>
    creator = 'AllanDaemon'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 41780
    keywords = ['patch']
    message_count = 8.0
    messages = ['376873', '376897', '376898', '376936', '376940', '376948', '376957', '376965']
    nosy_count = 5.0
    nosy_names = ['gvanrossum', 'docs@python', 'miss-islington', 'BTaskaya', 'AllanDaemon']
    pr_nums = ['22262', '22264']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue41780'
    versions = ['Python 3.9', 'Python 3.10']

    @AllanDaemon
    Copy link
    Mannequin Author

    AllanDaemon mannequin commented Sep 14, 2020

    The implementation of PEP-585 in 3.9 adds some new attributes, but they aren't listed with dir() (then, not list in autocomplete and IntelliSense).

    Python 3.9.0rc1+ (heads/3.9:d7cd1164c1, Aug 25 2020, 17:27:09)

    >>> li = list[int]
    >>> li.__origin__
    <class 'list'>
    >>> getattr(li, '__origin__')
    <class 'list'>
    >>> '__origin__' in dir(li)
    False

    That applies to:

    __origin__
    __args__
    __parameters__

    It seems to be not the expected behaviour, so this bug report. If this is not the case, some piece of documentation could be interesting. Also, I couldn't find anything about Generic Alias in the documentation, only in the source code. Should this be addressed too?

    @AllanDaemon AllanDaemon mannequin added docs Documentation in the Doc dir stdlib Python modules in the Lib dir labels Sep 14, 2020
    @AllanDaemon AllanDaemon mannequin assigned docspython Sep 14, 2020
    @AllanDaemon AllanDaemon mannequin added docs Documentation in the Doc dir 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 14, 2020
    @AllanDaemon AllanDaemon mannequin assigned docspython Sep 14, 2020
    @AllanDaemon AllanDaemon mannequin added 3.9 only security fixes 3.10 only security fixes type-bug An unexpected behavior, bug, or error labels Sep 14, 2020
    @gvanrossum
    Copy link
    Member

    I actually have no idea where dir() gets its information. IIRC the implementation was a collaboration with Batuhan -- @batuhan, do you want to look into this? Presumably some metadata needs to be added to genericaliasobject.c.

    @isidentical
    Copy link
    Sponsor Member

    @batuhan, do you want to look into this?

    Sure, will check it out!

    @AllanDaemon
    Copy link
    Mannequin Author

    AllanDaemon mannequin commented Sep 15, 2020

    I actually have no idea where dir() gets its information.

    I will share what I find out already, so it may hopefully be helpful.

    The dir function used is the general object.__dir__ function, defined in Objects/typeobject.c:type___dir___impl(), that just takes everything from __dict__ of the class and its bases.

    But the GenericAlias is trying hard pretending to be the origin class, so when you ask for the __dict__, it gives the original class, not the GenericAlias:

    >>> list[int].__dict__ == list.__dict__
    True

    There are also some other things that seems a bit strange, but it may just be right:

    >>> list.__class__
    <class 'type'>
    >>> list[int].__class__
    <class 'type'>
    
    >>> type(list)
    <class 'type'>
    >>> type(list[int])
    <class 'types.GenericAlias'>

    This happens because of the tp_getattro function genericaliasobject.c:ga_getattro(), that takes the attributes from the origin except for the following names, that are taken from the GenericAlias itself:
    "__origin__",
    "__args__",
    "__parameters__",
    "__mro_entries__",
    "__reduce_ex__",
    "__reduce__",

    So, when the dir() is getting the __dict__, it gets list.__dict__ and base (object) instead of GenericAlias. Looking the ga_getattro() calls when dir() is fired, it also gets the __class__ attribute after getting the __dict__. Adding one or another makes the dir() showing up the GenericAlias attributes, but others went missing.

    An idea to solve this spefic issue could be overriding the __dir__ method,
    something like this python code (written in c, of course):

    def __dir__(self):
        r = super().__dir__()
        return r + attribute_names_list + method_names_list

    But I wonder what else should not be forwarded to origin.

    @AllanDaemon AllanDaemon mannequin added interpreter-core (Objects, Python, Grammar, and Parser dirs) and removed docs Documentation in the Doc dir stdlib Python modules in the Lib dir labels Sep 15, 2020
    @isidentical
    Copy link
    Sponsor Member

    Okay, so IIRC if we stop forwarding __class__ (add it as an exception to attr_exceptions) it would return us the original dir(), and also solve the inconsistency of the example you gave;

    >>> list.__class__
    <class 'type'>
    >>> list[int].__class__
    <class 'type'>

    >>> type(list)
    <class 'type'>
    >>> type(list[int])
    <class 'types.GenericAlias'>

    but the problem is that, it is not 'exactly' complying with the specs at PEP-585. I am aware that we already added some extras to this list (https://www.python.org/dev/peps/pep-0585/#parameters-to-generics-are-available-at-runtime) like __mro_entries__ etc, but __class__ looks like a tricky problem. @gvanrossum any comments?

    @gvanrossum
    Copy link
    Member

    The forwarding of __class__, and the resulting difference between type(list[int]) and list[int].__class__ are intentional.

    I think overriding __dir__ along the lines suggested by Allan (but in C of course) would be the best solution.

    @miss-islington
    Copy link
    Contributor

    New changeset 2e87774 by Batuhan Taskaya in branch 'master':
    bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262)
    2e87774

    @miss-islington
    Copy link
    Contributor

    New changeset 49917d5 by Miss Islington (bot) in branch '3.9':
    bpo-41780: Fix __dir__ of types.GenericAlias (GH-22262)
    49917d5

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants