Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enum regression: AttributeError when accessing class variables on instances #87328

Closed
hroncok mannequin opened this issue Feb 8, 2021 · 13 comments
Closed

Enum regression: AttributeError when accessing class variables on instances #87328

hroncok mannequin opened this issue Feb 8, 2021 · 13 comments
Assignees
Labels
3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@hroncok
Copy link
Mannequin

hroncok mannequin commented Feb 8, 2021

BPO 43162
Nosy @ethanfurman, @hroncok
PRs
  • bpo-43162: [Enum] deprecate enum member.member access #24486
  • bpo-43162: [Enum] update docs, renable doc tests #24487
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ethanfurman'
    closed_at = <Date 2021-03-03.19:42:57.050>
    created_at = <Date 2021-02-08.12:22:59.718>
    labels = ['type-bug', 'library', '3.10']
    title = 'Enum regression: AttributeError when accessing class variables on instances'
    updated_at = <Date 2021-03-03.22:22:26.284>
    user = 'https://github.com/hroncok'

    bugs.python.org fields:

    activity = <Date 2021-03-03.22:22:26.284>
    actor = 'ethan.furman'
    assignee = 'ethan.furman'
    closed = True
    closed_date = <Date 2021-03-03.19:42:57.050>
    closer = 'ethan.furman'
    components = ['Library (Lib)']
    creation = <Date 2021-02-08.12:22:59.718>
    creator = 'hroncok'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 43162
    keywords = ['patch']
    message_count = 13.0
    messages = ['386626', '386628', '386642', '386668', '386672', '386674', '386680', '386683', '386700', '387953', '388038', '388053', '388063']
    nosy_count = 3.0
    nosy_names = ['ethan.furman', 'hroncok', 'baker.dylan.c']
    pr_nums = ['24486', '24487']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue43162'
    versions = ['Python 3.10']

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Feb 8, 2021

    I believe I found a regression in Enum in Python 3.10.0a5.

    This is Python 3.9:

    >>> import enum
    >>> class C(enum.Enum):
    ...     A = 0
    ...     B = 1
    ... 
    >>> C.A
    <C.A: 0>
    >>> C.B
    <C.B: 1>
    >>> C(0).A
    <C.A: 0>
    >>> C(0).B
    <C.B: 1>
    >>> 

    The Enum instances can access class-attributes via dot, like normal instances do.

    While in Python 3.10.0a5:

    >>> import enum
    >>> class C(enum.Enum):
    ...     A = 0
    ...     B = 1
    ... 
    >>> C.A
    <C.A: 0>
    >>> C.B
    <C.B: 1>
    >>> C(0).A
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib64/python3.10/enum.py", line 146, in __get__
        raise AttributeError(
    AttributeError: C: no attribute 'A'
    >>> C(0).B
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib64/python3.10/enum.py", line 146, in __get__
        raise AttributeError(
    AttributeError: C: no attribute 'B'

    In real word situations, it breaks meson:

    https://github.com/mesonbuild/meson/blob/398df5629863e913fa603cbf02c525a9f501f8a8/mesonbuild/backend/backends.py#L52-L78

    The __str__ method does:

        if self is self.EXITCODE: ...

    And it fails with:

    AttributeError: TestProtocol: no attribute 'EXITCODE'
    

    This worked with 3.10.0a4.

    If this is a deliberate backwards incompatible change of behavior, I don't think it is documented in the changelog or what's new in Python 3.10, nor that it was deprecated in Python 3.9 and 3.8.

    @hroncok hroncok mannequin added 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Feb 8, 2021
    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Feb 8, 2021

    Git bisect:

    c314e60 is the first new commit
    commit c314e60
    Author: Ethan Furman <ethan@stoneleaf.us>
    Date: Tue Jan 12 23:47:57 2021 -0800

    bpo-42901: [Enum] move member creation to `__set_name__` (GH-24196)
    
    `type.__new__` calls `__set_name__` and `__init_subclass__`, which means
    that any work metaclasses do after calling `super().__new__()` will not
    be available to those two methods.  In particular, `Enum` classes that
    want to make use of `__init_subclass__` will not see any members.
    
    Almost all customization is therefore moved to before the
    `type.__new__()` call, including changing all members to a proto member
    descriptor with a `__set_name__` that will do the final conversion of a
    member to be an instance of the `Enum` class.
    

    Lib/enum.py | 297 +++++++++++++--------
    Lib/inspect.py | 2 +-
    Lib/test/test_enum.py | 15 +-
    .../2021-01-11-17-36-59.bpo-42901.gFd-ta.rst | 3 +
    4 files changed, 207 insertions(+), 110 deletions(-)
    create mode 100644 Misc/NEWS.d/next/Library/2021-01-11-17-36-59.bpo-42901.gFd-ta.rst

    @ethanfurman
    Copy link
    Member

    The code for that __str__ seems very inefficient -- why doesn't it just do:

        return self.name

    ?

    -----

    The issue is not being able to access class attributes, the issue is whether one enum member should be seen as an attribute of another:

    EnumClass.RED.BLUE

    and the answer is no. That wasn't possible when Enum was first introduced in 3.4, and was an unfortunate side-effect of speeding up member access in 3.5 (or 3.6). The docs have always warned against it.

    A deprecation warning is an easier transition, though, so I'll do that for 3.10, and in 3.11 it will become an error.

    Thank you for reporting! :-)

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Feb 8, 2021

    Thanks. In the meantime, I've opened mesonbuild/meson#8318

    @bakerdylanc
    Copy link
    Mannequin

    bakerdylanc mannequin commented Feb 9, 2021

    Author of said meson code here. I use this pattern when the enum names and values are implementation details, but the string values are user visible. In this case the enum itself is used internally to represent what kind of test we're doing, but we're initializing it from user input. There might be reasons in the future that the names of the enum members and the string values being passed in aren't the same anymore, say because we map two string values to one enum value. I guess I could accomplish the same thing with a staticmethod or helper function, but the code is effectively an alternate initializer, and follows that pattern using a classmethod.

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Feb 9, 2021

    Ethan, should the depreciation exist for 2 releases prior to removal?

    Dylan, even in that case, I guess the proper way to access the other members is supposed to be type(self).FOO or ClassName.FOO, not self.FOO.

    @ethanfurman
    Copy link
    Member

    New changeset d65b903 by Ethan Furman in branch 'master':
    bpo-43162: [Enum] deprecate enum member.member access (GH-24486)
    d65b903

    @ethanfurman
    Copy link
    Member

    Dylan, it's not the from_str() method, but the __str__ method that is the problem. Instead of

        def __str__(self):
            if self is self.EXITCODE: 
                return 'exitcode' 

    it should be

        def __str__(self):
    
            cls = self.__class__
    
            if self is cls.EXITCODE: 
                return 'exitcode'

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Feb 9, 2021

    "Wait for the warning to appear in at least two major Python versions. It's fine to wait more than two releases."

    https://www.python.org/dev/peps/pep-0387/#basic-policy-for-backwards-compatibility

    So indeed, if you add the warning in 3.10, the behavior can be removed from 3.12 earliest.

    @ethanfurman
    Copy link
    Member

    DeprecationWarning will be active in 3.10 and 3.11 with removal in 3.12.

    @ethanfurman
    Copy link
    Member

    New changeset 44e580f by Ethan Furman in branch 'master':
    bpo-43162: [Enum] update docs, renable doc tests (GH-24487)
    44e580f

    @hroncok
    Copy link
    Mannequin Author

    hroncok mannequin commented Mar 3, 2021

    Thank you, Ethan.

    @ethanfurman
    Copy link
    Member

    You're welcome. Thank you for pushing the issue! :-)

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant