This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Enum equality across modules: comparing objects instead of values
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: Madhav Datt, Markus Wegmann, Sebastian Höfer, adrianwan2, ethan.furman, r.david.murray
Priority: normal Keywords:

Created on 2017-06-02 04:58 by Madhav Datt, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
enum_bug-flawless_example.zip Markus Wegmann, 2018-08-11 09:25 Structurally similar example program running flawlessly
enum_reload_example.zip Sebastian Höfer, 2018-11-16 13:05
Messages (11)
msg294983 - (view) Author: Madhav Datt (Madhav Datt) Date: 2017-06-02 04:58
The problem is described with an example in this StackOverflow question (https://stackoverflow.com/questions/26589805/python-enums-across-modules). Like in C and other languages, I would expect Enum equality to work across modules and not compare enum states/values, instead of just checking for the same object.

A possible simple fix for this problem would be to override the __eq__() function by default in the enum.Enum class with the following:

def __eq__(self, other):
    if isinstance(other, self.__class__):
        return self.value == other.value
    return False

I would be happy to create a GitHub pull request to fix this, however, I do not have the experience or knowledge to know if
- the current behavior is by design;
- whether this is worth fixing; and
- whether fixing this will break anything else.
msg295025 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-06-02 15:23
Two points:

- Python 2.7 was the version marked, but 2.7 does not come with Enum
  (wasn't introduced until 3.4 -- the third-party backport does work
  on 2.7)

- the problem in the SO question is not caused by Enum, but by
  re-importing a module under a different name which results in two
  different Enum classes that happen to look identical, but are not --
  so the change you propose would not help; also, since Enum members
  with the same value are mapped to the same member your change does not
  provide any new behavior.

So, in summary, the bug here is in the user's code.
msg295032 - (view) Author: Madhav Datt (Madhav Datt) Date: 2017-06-02 16:13
Thanks a lot for those points Ethan. I feel I haven't done a very good job of explaining the bug, but let me use an example. Let's say we have an Enum called MyEnum, which is in a Python module called ModuleA. ModuleB imports ModuleA, and ModuleC imports both, ModuleA and ModuleB. Now, in ModuleC, I have ModuleB.some_function() return a MyEnum state, which I pass as a parameter to ModuleA.other_function() where it is compared to MyEnum states. Here the comparison fails even though it should not have. Obviously, this problem would not arise without such imports, and so is pretty specific, but I hope this makes explains it a little better.
msg295036 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-06-02 16:35
Can you provide actual code that demonstrates the issue you are talking about?
msg295038 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-06-02 16:45
If your example code is the same as the code in the SO problem, then my previous points stand.

According to the plain-English description you provided the comparison would succeed, so if you have example code which:

- doesn't involve ModuleA being the __main__ script module, and
- has the comparison fail, then

please share it.  ;)  (As a comment/message here is fine.)
msg295457 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2017-06-08 18:01
No test code has been provided, so lacking any evidence of this problem I am closing this issue.

Do not reopen without testable code to show the failure.
msg323411 - (view) Author: Markus Wegmann (Markus Wegmann) Date: 2018-08-11 09:25
Hi there!

I came as well in contact with this kind of bug. Sadly, I could not replicate it with a simplified toy example.

You can experience the bug, if checkout

https://github.com/Atokulus/flora_tools/tree/56bb17ea33c910915875214e916ab73f567b3b0c

and run

`python3.7 main.py generate_code -d ..`

You find the crucial part where the identity check fails at 'radio_math.py:102' in function `get_message_toa`. The program then fails, due to the wrong program flow.

`
C:\Users\marku\AppData\Local\Programs\Python\Python37\python.exe C:/Users/marku/PycharmProjects/flora_tools/flora_tools/__main__.py generate_code -d C:\Users\marku\Documents\flora
Traceback (most recent call last):
  File "C:/Users/marku/PycharmProjects/flora_tools/flora_tools/__main__.py", line 110, in <module>
    main()
  File "C:/Users/marku/PycharmProjects/flora_tools/flora_tools/__main__.py", line 104, in main
    generate_code(args.path)
  File "C:/Users/marku/PycharmProjects/flora_tools/flora_tools/__main__.py", line 58, in generate_code
    code_gen = CodeGen(flora_path)
  File "C:\Users\marku\PycharmProjects\flora_tools\flora_tools\codegen\codegen.py", line 37, in __init__
    self.generate_all()
  File "C:\Users\marku\PycharmProjects\flora_tools\flora_tools\codegen\codegen.py", line 40, in generate_all
    self.generate_radio_constants()
  File "C:\Users\marku\PycharmProjects\flora_tools\flora_tools\codegen\codegen.py", line 72, in generate_radio_constants
    radio_toas.append([math.get_message_toa(payload) for payload in payloads])
  File "C:\Users\marku\PycharmProjects\flora_tools\flora_tools\codegen\codegen.py", line 72, in <listcomp>
    radio_toas.append([math.get_message_toa(payload) for payload in payloads])
  File "C:\Users\marku\PycharmProjects\flora_tools\flora_tools\radio_math.py", line 130, in get_message_toa
    ) / self.configuration.bitrate
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Process finished with exit code 1
`

In the appendix, you find a toy project with similar structure, which in contrast to `flora_tools` works flawlessly.

I hope there is somebody out there with a little more experience in spotting the cause. If there is a bug in Python or CPython, it might have quite a security & safety impact.

Meanwhile I will hotfix my program by comparing the unerlying enum values.

Best regards
Atokulus
msg323418 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2018-08-11 17:31
Markus Wegmann said:
--------------------
> In the appendix, you find a toy project with similar structure, which > in contrast to `flora_tools` works flawlessly.

I appreciate your attempt to reproduce the problem, but since you weren't able to, the issue is probably somewhere else and not in Enum.

> TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Your Enum example in flawless is not an IntEnum, so the error (unable to add an integer to None) seems entirely unrelated.
msg323432 - (view) Author: Markus Wegmann (Markus Wegmann) Date: 2018-08-12 06:33
Hi Ethan

> Your Enum example in flawless is not an IntEnum, so the error (unable to add an integer to None) seems entirely unrelated.

The TypeError is just a consequence of the faulty Enum identity comparison some lines before. I mentioned the TypeError so you can verify whether your Python version takes the same program flow.


I also did further research. The Enum's are definitely different regarding the module path -- the instance comparison will therefore return False. I checked __module__ + __qualname__:

`flora_tools.radio_configuration.RadioModem.LORA`

vs.

`radio_configuration.RadioModem.LORA`


The cause is the wrong import statement in `flora_tools/codegen/codegen.py`:

`from radio_configuration import RadioConfiguration,\ 
                                RADIO_CONFIGURATIONS`

It should have been

`from flora_tools.radio_configuration import RadioConfiguration\
                                            RADIO_CONFIGURATIONS`

The real deal here is why I was allowed to directly import from `radio_configuration` in the first place. I'm not allowed to directly import a submodule in the toy project without the proper root module name appended. Maybe I don't see the big picture, or have some crude options/monkey_patching enabled.

Nevertheless, the behaviour regarding Enum comparisons and different import paths seems to me quite misleading.

Best regards
Atokulus
msg323433 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2018-08-12 07:10
What you have discovered is not Enum specific, but in fact can happen with any module (as stated in my first comment in this bug report).

Maybe these SO question will help:

https://stackoverflow.com/q/2489601/208880

https://stackoverflow.com/a/4798648/208880
msg329994 - (view) Author: Sebastian Höfer (Sebastian Höfer) Date: 2018-11-16 13:05
I ran into the same issue and while I see that this is not a bug I would suggest that this behaviour at least deserves a warning in the documentation under
https://docs.python.org/3/library/enum.html#comparisons

This unexpected behaviour can not only occur with messed up imports, but also when a module uses the importlib.reload() function and submodules are reloaded during runtime.

I attached a minimal working example. 
In our case we have a central module where all constants, dataclasses and enums are defined. A main program imports several submodules which all import these datatypes. Reloading a submodule results in new object ids for the enums in this submodule and comparing data between different modules suddenly results in inconsistent behaviour.

It took a while to debug, because except for the object id the enums look exactly the same and before reading this thread I did not realize that the comparison is actually done by object ids.
History
Date User Action Args
2022-04-11 14:58:47adminsetgithub: 74730
2018-11-16 13:05:55Sebastian Höfersetfiles: + enum_reload_example.zip
nosy: + Sebastian Höfer
messages: + msg329994

2018-08-12 07:10:06ethan.furmansetmessages: + msg323433
2018-08-12 06:33:10Markus Wegmannsetmessages: + msg323432
2018-08-11 17:31:39ethan.furmansetmessages: + msg323418
2018-08-11 09:25:15Markus Wegmannsetfiles: + enum_bug-flawless_example.zip
nosy: + Markus Wegmann
messages: + msg323411

2017-06-08 18:01:08ethan.furmansetstatus: open -> closed
resolution: not a bug
messages: + msg295457

stage: test needed -> resolved
2017-06-05 21:06:20adrianwan2setstatus: pending -> open
nosy: + adrianwan2
2017-06-03 16:38:38serhiy.storchakasetstatus: open -> pending
2017-06-02 16:45:19ethan.furmansetresolution: not a bug -> (no value)
messages: + msg295038
stage: resolved -> test needed
2017-06-02 16:35:01r.david.murraysetnosy: + r.david.murray

messages: + msg295036
versions: - Python 2.7
2017-06-02 16:13:02Madhav Dattsetstatus: closed -> open

messages: + msg295032
2017-06-02 15:23:11ethan.furmansetstatus: open -> closed
versions: + Python 3.7
messages: + msg295025

resolution: not a bug
stage: resolved
2017-06-02 07:14:01rhettingersetassignee: ethan.furman

nosy: + ethan.furman
2017-06-02 04:58:59Madhav Dattcreate