classification
Title: assertRaises with exceptions re-raised from a generator kills generator
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: dalke, epiphyte
Priority: normal Keywords:

Created on 2017-01-08 21:14 by dalke, last changed 2020-05-05 19:17 by epiphyte.

Messages (1)
msg285006 - (view) Author: Andrew Dalke (dalke) * (Python committer) Date: 2017-01-08 21:14
The unittest assertRaises/assertRaisesRegex implementation calls traceback.clear_frames() because of issue9815 ("assertRaises as a context manager keeps tracebacks and frames alive").

However, if the traceback is from an exception created in a generator, caught, and re-raised outside of the generator, then the clear_frames() will cause the generator to raise a StopIteration exception the next time it is used.

Here is a reproducible where I create a generator and wrap it inside of an object API:

def simple_gen():
    yield 1, None
    try:
        1/0
    except ZeroDivisionError as err:
        yield None, err
    yield 3, None

class Spam:
    def __init__(self):
        self.gen = simple_gen()
    def get_next(self):
        value, err = next(self.gen)
        if err is not None:
            raise err
        return value

I can test this without unittest using the following:

def simple_test():
    spam = Spam()
    assert spam.get_next() == 1
    try:
        spam.get_next()
    except ZeroDivisionError:
        pass
    else:
        raise AssertionError
    assert spam.get_next() == 3
    print("simple test passed")

simple_test()


This prints "simple test passed", as expected.

The unittest implementation is simpler:

import unittest

class TestGen(unittest.TestCase):
    def test_gen(self):
        spam = Spam()
        self.assertEqual(spam.get_next(), 1)
        with self.assertRaises(ZeroDivisionError):
            spam.get_next()
        self.assertEqual(spam.get_next(), 3)

unittest.main()

but it reports an unexpected error:

======================================================================
ERROR: test_gen (__main__.TestGen)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "clear.py", line 40, in test_gen
   self.assertEqual(spam.get_next(), 3)
 File "clear.py", line 13, in get_next
   value, err = next(self.gen)
StopIteration

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

I have tracked it down to the call to traceback.clear_frames(tb) in unittest/case.py. The following ClearFrames context manager will call traceback.clear_frames() if requested. The test code uses ClearFrames to demonstrate that the call to clear_frames() is what causes the unexpected StopIteration exception:


import traceback

class ClearFrames:
   def __init__(self, clear_frames):
       self.clear_frames = clear_frames
   def __enter__(self):
       return self

   def __exit__(self, exc_type, exc_value, tb):
       assert exc_type is ZeroDivisionError, exc_type
       if self.clear_frames:
           traceback.clear_frames(tb)  # This is the only difference between the tests.
       return True

# This is essentially the same test case as before, but structured using
# a context manager that either does or does not clear the traceback frames.
def clear_test(clear_frames):
    spam = Spam()
    assert spam.get_next() == 1
    with ClearFrames(clear_frames):
        spam.get_next()
    try:
        assert spam.get_next() == 3
    except StopIteration:
        print(" ... got StopIteration")
        return
    print(" ... clear_test passed")

print("\nDo not clear frames")
clear_test(False)
print("\nClear frames")
clear_test(True)


The output from this test is:

Do not clear frames
 ... clear_test passed

Clear frames
 ... got StopIteration

There are only a dozen or so tests in my code which are affected by this. (These are from a test suite which I am porting from 2.7 to 3.5.) I can easily re-write them to avoid using assertRaisesRegex.

I have no suggestion for a longer-term solution.
History
Date User Action Args
2020-05-05 19:17:23epiphytesetnosy: + epiphyte
2017-01-08 21:14:24dalkecreate