Stepping back slightly, it is more general to say that str, and in certain other cases dict and set (and possibly others) will raise instead of return False when it is impossible for the target type to ever hold the checked-for type.  A couple examples of what will raise:

    1 in 'hello'       # integers will never be in a string
    list() in dict()   # dict keys must be hashable (and lists are not)

So, yes, at least for pure Enums and Flags, raising TypeError when a non-Enum/Flag is checked for would be appropriate.

Since there may be code currently relying on always getting True/False, though, a deprecation period is called for.  I'll see if I can get that into 3.7.