When a unittest test case raises an Exception, that test case is
considered a failure. However, raising NotImplementedError is not a
failure. It is something completely normal during development and 
simply indicates that the functionality has not yet been implemented.
Compare this to a test case that genuinely fails, which means that the
implementation does something different from what is is supposed to do.

I therefore think test cases raising NotImplementedError should not be
treated as failures, but be reported separately as what they are --
parts of the program that have not yet been implemented. Reporting such 
test cases as failures will give a lot of failure warnings that may 
detract attention from those cases that are genuinely errors. Also, 
SimpleTextRunner will report a lot of useless stack traces for these 
cases, thereby makeing it harder to find the stack traces for those
tests that realy failed.

I have made a patch to unittest that reports such cases as "not 
implemented" rather than as failures. This includes changes to the 
documentation and unit test case for unittest and doctest. 
I have included 
"added/changed in version 2.6" remarks in the documentation, so
you have to changed these if the patch gets into some other version.

Patch against 59056 attached

No. If you're testing something that's not implemented, it is correct to
see that as a failure. Talk to anyone doing TDD.

I don't think unittest automatically treats all exceptions as failures.
Failures are those that are explicitly flagged with assert* and fail*
methods. All other exceptions result in "errors". I think what you are
asking for is to special case one such error but am not sure if it is
worth it.

Raghuram Devarakonda skrev:
> Raghuram Devarakonda added the comment:
>
> I don't think unittest automatically treats all exceptions as failures.
> Failures are those that are explicitly flagged with assert* and fail*
> methods. All other exceptions result in "errors". I think what you are
> asking for is to special case one such error but am not sure if it is
> worth it.
>
> ----------
> nosy: +draghuram
>
> __________________________________
> Tracker <report@bugs.python.org>
> <http://bugs.python.org/issue1466>
> __________________________________
>   

You are right, I used the wrong word.  I mean that NotImplementedError 
should be treated as a special *error*.

If you have ever written the test cases before you write the code, you 
will get a huge amount of uninteresting output from
errors that come from tests to code you havn't written yet. This will 
drown the real errors and failures from the code
you are actually testing and debugging. What I want is to have unittest 
report these differently so that I can see what are the
real bugs the code, not
to spam me with tracebacks I will never look at for test cases that 
refer to not implemented code.

Erik

Sorry, just comment out those tests until you're ready to write the code.

Possible, but cumbersome. You have to analyze each case and see if it 
depends on code not written.
When you implement a new piece of code, you have to go through all your 
tests to see if that one should
be included.
Also you don't get any reporting on the coverage of the code.
If you accidentally forget to uncomment a test case, you may never 
realize that you have a bug --
at least not until it happens in real life.

Erik

> Possible, but cumbersome. You have to analyze each case and see if it

Have you tried to write a Test Runner that ignores
NotImplementedError? You may have to override TestResult.addError.

Sorry Erik, but I don't think you should try to compensate for your
flawed methodology by trying to change the unittest framework. End of
discussion.

No, I definitely don't want to ignore them, I want them reported in a 
manner that is relevant to them. Just say that the test was not implemented.
No tracebacks for debugging since there is nothing to debug.

In addition to a TestRunner I need a new TestCase and a new TestResult, 
which is what is in the patch,
although I changed the classes rather than inherit from them.

Erik