While `Flag() in Flag()` and `Flag() | Flag()` result in the expected outcomes, e.g. `str() in Flag()` unexpectedly returns `True`, whereas `Flag() | str()` as expected raises a TypeError.
    >>> import enum
    >>> ABC = enum.Flag('ABC', 'a, b, c')
    >>> ac = ABC.a | ABC.c
    >>> def test():
    ...     for x in (*ABC, 'test'):
    ...         print(x, end=' ')
    ...         try:
    ...             print(x in ac, end=' ')
    ...         except TypeError as e:
    ...             print(e, end=' ')
    ...         try:
    ...             print(x | ac)
    ...         except TypeError as e:
    ...             print(e)
    >>> test()
    ABC.a True ABC.c|a
    ABC.b False ABC.c|b|a
    ABC.c True ABC.c|a
    test True unsupported operand type(s) for |: 'str' and 'ABC'

Likely cause is modelling of Flag.contains after Flag.and etc., which is incorrect as contains doesn't have a reflected version like and etc. have. The returned NotImplemented is therefore not handled by the interpreter, although it is converted to bool to satisfy contains return type.

This can be fixed by redefinition of Flag.__contains__ to raise TypeError:
    >>> def __contains__(self, other):
    ...     if not isinstance(other, self.__class__):
    ...         raise TypeError(f"unsupported operand type(s) for 'in': "
    ...                         f"{type(other).__qualname__!r} and {type(self).__qualname__!r}")
    ...     return other & self == other
    >>> enum.Flag.__contains__ = __contains__
    >>> test()
    ABC.a True ABC.c|a
    ABC.b False ABC.c|b|a
    ABC.c True ABC.c|a
    test unsupported operand type(s) for 'in': 'str' and 'ABC' unsupported operand type(s) for |: 'str' and 'ABC'

Thanks for finding this!

However, TypeError, an exception, is not the correct type of answer -- the correct type of answer for a yes/no question is either True or False.

So the code should be changed from returning NotImplemented to returning False.

Fully agree that Flag.contains must RETURN False or True; that's why I suggested it RAISES TypeError
The exception was to be consistent with e.g. Flag.and, which by returning NotImplemented transfers to type(other).rand, and assuming rand cannot handle Flag and also returns NotImplemented, falls back to the interpreter raising TypeError. Also e.g. 1 in 'hello' raises TypeError.
For Flag, I wouldn't strongly oppose to False for other being a different type. However, classes derived from Flag could face issues, e.g. class MyIntFlag (Flag, int): pass.

Ah, the mixin logic mentioned in 33219 solves most of my objections to returning False; agree that would make sense
Only remaining inconsistency would be with 1 in 'hello'

Strings are actually the odd-man out -- dicts, sets, lists, tuples, etc., all return False instead of raising TypeError.

The reason str raises an error is because in, for str, is a substring check, not a membership check.

Stepping back slightly, it is more general to say that str, and in certain other cases dict and set (and possibly others) will raise instead of return False when it is impossible for the target type to ever hold the checked-for type. A couple examples of what will raise:

1 in 'hello'       # integers will never be in a string
list() in dict()   # dict keys must be hashable (and lists are not)

So, yes, at least for pure Enums and Flags, raising TypeError when a non-Enum/Flag is checked for would be appropriate.

Since there may be code currently relying on always getting True/False, though, a deprecation period is called for. I'll see if I can get that into 3.7.

New changeset 3715176 by Ethan Furman in branch '3.7':
[3.7] bpo-33217: deprecate non-Enum lookups in Enums (GH-6392)
3715176

DeprecationWarning is in 3.7, now need to raise TypeError in 3.8.

Hi Ethan,

The only thing which is left is to change the Deprecation Warning to raise a TypeError and alter the tests accordingly.

Seeing that most of the work for the issue has already been done, can I take it forward from here on wards, please?

Rahul Jha, sure, go ahead!

New changeset 9430652 by Ethan Furman (Rahul Jha) in branch 'master':
bpo-33217: Raise TypeError for non-Enum lookups in Enums (GH-6651)
9430652

Thank you, Rahul!

I use python 3.8.0a3 to make our testframework ready for the future.

Is it volitional that the Expression "if int in IntEnum:" raise also a TypeError?

A test for this has been added for IntFlag so I think it must have been on purpose : 9430652#diff-d57e55a3bb4873aec10786e531a88947R2386