This is a common mistake. Even the default implementation of object.__ne__ had a bug, fixed only 4 years ago in issue21408. There were several errors in stdlib. I am sure there is a lot of occurrences of this errors in a third party code.

So I like this idea. This can help to fix hidden errors in existing code. But I share also Josh's concerns.

There is related common mistake. In C code, the result of PyObject_IsTrue() often is treated as just a boolean, without checking for errors. Fortunately, in the current CPython code it is handled properly, but I am sure this error still is occurred in third-party extensions.

When these two errors (using NotImplemented in the boolean context and ignoring the error in PyObject_IsTrue() in the C code) meet, this can lead to manifestation of more weird bugs than treating NotImplemented as true: from a crash in debug build to raising an exception in the following unrelated code.

I suggest to start emitting a DeprecationWarning or a FutureWarning in NotImplemented.__bool__.