Title: enum.Flag ~ bitwise negation is very slow and can't be defined as a Flag value
Type: behavior Stage: resolved
Components: Versions: Python 3.8
Status: closed Resolution: wont fix
Dependencies: Superseder: enum.Flag should be more set-like
View: 38250
Assigned To: ethan.furman Nosy List: aspin2, ethan.furman
Priority: normal Keywords:

Created on 2021-01-12 23:31 by aspin2, last changed 2021-01-25 23:11 by ethan.furman. This issue is now closed.

Messages (5)
msg384983 - (view) Author: Kevin Chen (aspin2) Date: 2021-01-12 23:31
Here's a code sample:

import time
from enum import Flag, auto

class MyFlag(Flag):
    NONE = 0
    FLAG_1 = auto()
    FLAG_2 = auto()
    FLAG_3 = auto()
    FLAG_4 = auto()
    FLAG_5 = auto()
    FLAG_6 = auto()
    # NOT_FLAG_1_OR_2 = ~FLAG_1 & ~FLAG_2

def test_flag():
    f = MyFlag.NONE
    inverted = (
        & ~MyFlag.FLAG_2
        & ~MyFlag.FLAG_3
        & ~MyFlag.FLAG_4
        & ~MyFlag.FLAG_5
        & ~MyFlag.FLAG_6
    return f & inverted

    & ~MyFlag.FLAG_2
    & ~MyFlag.FLAG_3
    & ~MyFlag.FLAG_4
    & ~MyFlag.FLAG_5
    & ~MyFlag.FLAG_6

def test_flag_cached():
    f = MyFlag.NONE
    return f & INVERTED

if __name__ == "__main__":
    start_time = time.time()
    for _ in range(10_000):

    elapsed = time.time() - start_time
    print(f"Took normal {elapsed:2f} seconds.")

    start_time = time.time()
    for _ in range(10_000):

    elapsed = time.time() - start_time
    print(f"Took cached {elapsed:2f} seconds.")

And its outputs:
Took normal 1.799731 seconds.
Took cached 0.009488 seconds.

Basically, bitwise negation is very very slow. From what I can tell, it seems that a lot of time is spent here computing powers of two. I've read elsewhere that flag values are cached, and it looks like negated Flag values can't be cached? This seems related to the second issue, which is that any negated Flag value being defined results in `RecursionError: maximum recursion depth exceeded` as it searches for a good name for Flag.

Obviously, the simple workaround is just to define a constant variable elsewhere with the negated value, but it isn't very obvious anywhere that this is necessary, and I wanted to raise this to see if anyone has knowledge of the implementation details of Flag for possibly resolving this in the class itself.
msg385004 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-01-13 07:42
I just finished a rewrite of Flag for 3.10.  Using your test below I was able to tweak the rewrite so the final numbers are:

Took normal 0.148092 seconds.
Took cached 0.017438 seconds.

Your original post had a ratio of nearly 200 -- it is now 8.7ish.
msg385054 - (view) Author: Kevin Chen (aspin2) Date: 2021-01-13 21:27
Awesome thanks! Does the rewrite fix the issue with creating negated flags as well?
msg385057 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-01-13 22:52
msg385678 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-01-25 23:11
Fixed in 3.10 in issue38250.

Also fixed in my 3rd-party library, aenum 3.0:

Date User Action Args
2021-01-25 23:11:52ethan.furmansetstatus: open -> closed
superseder: enum.Flag should be more set-like
messages: + msg385678

type: behavior
resolution: wont fix
stage: resolved
2021-01-13 22:52:22ethan.furmansetmessages: + msg385057
2021-01-13 21:27:32aspin2setmessages: + msg385054
2021-01-13 07:42:09ethan.furmansetassignee: ethan.furman
messages: + msg385004
2021-01-13 03:54:18gvanrossumsetnosy: + ethan.furman
2021-01-12 23:31:13aspin2create