classification
Title: Enum regression: AttributeError when accessing class variables on instances
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: baker.dylan.c, ethan.furman, hroncok
Priority: normal Keywords: patch

Created on 2021-02-08 12:22 by hroncok, last changed 2021-03-03 22:22 by ethan.furman. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 24486 merged ethan.furman, 2021-02-09 00:32
PR 24487 merged ethan.furman, 2021-02-09 02:45
Messages (13)
msg386626 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-02-08 12:22
I believe I found a regression in Enum in Python 3.10.0a5.

This is Python 3.9:

>>> import enum
>>> class C(enum.Enum):
...     A = 0
...     B = 1
... 
>>> C.A
<C.A: 0>
>>> C.B
<C.B: 1>
>>> C(0).A
<C.A: 0>
>>> C(0).B
<C.B: 1>
>>> 


The Enum instances can access class-attributes via dot, like normal instances do.


While in Python 3.10.0a5:

>>> import enum
>>> class C(enum.Enum):
...     A = 0
...     B = 1
... 
>>> C.A
<C.A: 0>
>>> C.B
<C.B: 1>
>>> C(0).A
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.10/enum.py", line 146, in __get__
    raise AttributeError(
AttributeError: C: no attribute 'A'
>>> C(0).B
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.10/enum.py", line 146, in __get__
    raise AttributeError(
AttributeError: C: no attribute 'B'





In real word situations, it breaks meson:

https://github.com/mesonbuild/meson/blob/398df5629863e913fa603cbf02c525a9f501f8a8/mesonbuild/backend/backends.py#L52-L78

The __str__ method does:

    if self is self.EXITCODE: ...

And it fails with:

    AttributeError: TestProtocol: no attribute 'EXITCODE'

This worked with 3.10.0a4.


If this is a deliberate backwards incompatible change of behavior, I don't think it is documented in the changelog or what's new in Python 3.10, nor that it was deprecated in Python 3.9 and 3.8.
msg386628 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-02-08 12:41
Git bisect:

c314e60388282d9829762fb6c30b12e2807caa19 is the first new commit
commit c314e60388282d9829762fb6c30b12e2807caa19
Author: Ethan Furman <ethan@stoneleaf.us>
Date:   Tue Jan 12 23:47:57 2021 -0800

    bpo-42901: [Enum] move member creation to `__set_name__` (GH-24196)
    
    `type.__new__` calls `__set_name__` and `__init_subclass__`, which means
    that any work metaclasses do after calling `super().__new__()` will not
    be available to those two methods.  In particular, `Enum` classes that
    want to make use of `__init_subclass__` will not see any members.
    
    Almost all customization is therefore moved to before the
    `type.__new__()` call, including changing all members to a proto member
    descriptor with a `__set_name__` that will do the final conversion of a
    member to be an instance of the `Enum` class.

 Lib/enum.py                                        | 297 +++++++++++++--------
 Lib/inspect.py                                     |   2 +-
 Lib/test/test_enum.py                              |  15 +-
 .../2021-01-11-17-36-59.bpo-42901.gFd-ta.rst       |   3 +
 4 files changed, 207 insertions(+), 110 deletions(-)
 create mode 100644 bpo-42901.gFd-ta.rst">Misc/NEWS.d/next/Library/2021-01-11-17-36-59.bpo-42901.gFd-ta.rst
msg386642 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-02-08 18:32
The code for that `__str__` seems very inefficient -- why doesn't it just do:

    return self.name

?

-----

The issue is not being able to access class attributes, the issue is whether one enum member should be seen as an attribute of another:

  EnumClass.RED.BLUE

and the answer is no.  That wasn't possible when Enum was first introduced in 3.4, and was an unfortunate side-effect of speeding up member access in 3.5 (or 3.6).  The docs have always warned against it.

A deprecation warning is an easier transition, though, so I'll do that for 3.10, and in 3.11 it will become an error.

Thank you for reporting! :-)
msg386668 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-02-08 22:09
Thanks. In the meantime, I've opened https://github.com/mesonbuild/meson/issues/8318
msg386672 - (view) Author: Dylan Baker (baker.dylan.c) Date: 2021-02-09 00:43
Author of said meson code here. I use this pattern when the enum names and values are implementation details, but the string values are user visible. In this case the enum itself is used internally to represent what kind of test we're doing, but we're initializing it from user input. There might be reasons in the future that the names of the enum members and the string values being passed in aren't the same anymore, say because we map two string values to one enum value. I guess I could accomplish the same thing with a staticmethod or helper function, but the code is effectively an alternate initializer, and follows that pattern using a classmethod.
msg386674 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-02-09 01:00
Ethan, should the depreciation exist for 2 releases prior to removal?

Dylan, even in that case, I guess the proper way to access the other members is supposed to be type(self).FOO or ClassName.FOO, not self.FOO.
msg386680 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-02-09 01:33
New changeset d65b9033d6d092552775f6f5e41e7647100f9f2c by Ethan Furman in branch 'master':
bpo-43162: [Enum] deprecate enum member.member access (GH-24486)
https://github.com/python/cpython/commit/d65b9033d6d092552775f6f5e41e7647100f9f2c
msg386683 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-02-09 01:43
Dylan, it's not the `from_str()` method, but the `__str__` method that is the problem.  Instead of

    def __str__(self):
        if self is self.EXITCODE: 
            return 'exitcode' 

it should be

    def __str__(self):

        cls = self.__class__

        if self is cls.EXITCODE: 
            return 'exitcode'
msg386700 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-02-09 08:58
"Wait for the warning to appear in at least two major Python versions. It's fine to wait more than two releases."

https://www.python.org/dev/peps/pep-0387/#basic-policy-for-backwards-compatibility

So indeed, if you add the warning in 3.10, the behavior can be removed from 3.12 earliest.
msg387953 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-03-02 19:05
DeprecationWarning will be active in 3.10 and 3.11 with removal in 3.12.
msg388038 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-03-03 17:54
New changeset 44e580f448016b86807465a186d03d9074e2b589 by Ethan Furman in branch 'master':
bpo-43162: [Enum] update docs, renable doc tests (GH-24487)
https://github.com/python/cpython/commit/44e580f448016b86807465a186d03d9074e2b589
msg388053 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-03-03 20:13
Thank you, Ethan.
msg388063 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-03-03 22:22
You're welcome.  Thank you for pushing the issue!  :-)
History
Date User Action Args
2021-03-03 22:22:26ethan.furmansetmessages: + msg388063
2021-03-03 20:13:22hroncoksetmessages: + msg388053
2021-03-03 19:42:57ethan.furmansetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-03-03 17:54:37ethan.furmansetmessages: + msg388038
2021-03-02 19:05:09ethan.furmansetmessages: + msg387953
2021-02-09 08:58:18hroncoksetmessages: + msg386700
2021-02-09 02:45:15ethan.furmansetpull_requests: + pull_request23277
2021-02-09 01:43:24ethan.furmansetmessages: + msg386683
2021-02-09 01:33:00ethan.furmansetmessages: + msg386680
2021-02-09 01:00:37hroncoksetmessages: + msg386674
2021-02-09 00:43:36baker.dylan.csetnosy: + baker.dylan.c
messages: + msg386672
2021-02-09 00:32:48ethan.furmansetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request23276
2021-02-08 22:09:57hroncoksetmessages: + msg386668
2021-02-08 18:32:58ethan.furmansetassignee: ethan.furman
messages: + msg386642
stage: needs patch
2021-02-08 12:41:12hroncoksetnosy: + ethan.furman
messages: + msg386628
2021-02-08 12:22:59hroncokcreate