classification
Title: Unit tests that return generators silently fail
Type: Stage: resolved
Components: None Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Jason.Baker, alex, ezio.melotti, michael.foord, r.david.murray
Priority: normal Keywords:

Created on 2012-08-03 18:08 by Jason.Baker, last changed 2012-08-04 21:54 by r.david.murray. This issue is now closed.

Messages (8)
msg167340 - (view) Author: Jason Baker (Jason.Baker) Date: 2012-08-03 18:08
The following test will pass silently:


    class SomeTest(unittest.TestCase):

      def testSomething(self):
        yield 1
        self.fail()
msg167351 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-08-03 20:25
unittest calls test methods to execute tests. Generator functions do nothing (except create a generator) when you call them. Test methods as generators therefore do nothing in unittest.
msg167354 - (view) Author: Jason Baker (Jason.Baker) Date: 2012-08-03 21:09
I can play this game too!

Since generator functions do nothing when used as a unittest test method, there is no value in having a test method that is a generator function.  Therefore, it is an error to give a test method that is a generator function.  Since it is an error to have a generator function as a test method, the test I gave should fail.
msg167356 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-08-03 21:10
It's not a game. Have a test method as a generator function is clearly programmer error. Doing nothing is the expected behaviour.
msg167360 - (view) Author: Jason Baker (Jason.Baker) Date: 2012-08-03 21:24
Who expects this behavior?  Can you give me an example of when someone would rely upon a generator test to do nothing?
msg167361 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2012-08-03 21:25
It's the expected behaviour according to the unittest specification.
msg167363 - (view) Author: Jason Baker (Jason.Baker) Date: 2012-08-03 21:31
I would like to verify this.  Link the specification, and highlight where it says that the expected behavior for a test method that is a generator function is to do nothing.
msg167447 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-08-04 21:54
This is not something that is specific to unittest.  In Python, if you call a generator function *it returns a generator-iterator*.  Unless you *do* something with the the iterator, nothing else happens.  This is true in *any* python code.  

Unittest calls whatever test method you define, and handles (reports) the exceptions that result from that call.  That's the fundamental design of unittest.  Your generator test method does not raise any exceptions when called, therefore the test passed.
History
Date User Action Args
2012-08-04 21:54:38r.david.murraysetnosy: + r.david.murray
messages: + msg167447
2012-08-03 21:31:57Jason.Bakersetmessages: + msg167363
2012-08-03 21:25:30michael.foordsetmessages: + msg167361
2012-08-03 21:24:49Jason.Bakersetmessages: + msg167360
2012-08-03 21:10:00michael.foordsetmessages: + msg167356
2012-08-03 21:09:11Jason.Bakersetmessages: + msg167354
2012-08-03 20:25:37michael.foordsetstatus: open -> closed
resolution: not a bug
messages: + msg167351

stage: resolved
2012-08-03 20:23:01ned.deilysetnosy: + ezio.melotti, michael.foord
2012-08-03 18:08:58alexsetnosy: + alex
2012-08-03 18:08:18Jason.Bakercreate