David is right. The 120 code was added in Issue 5319, as a way of indicating a problem in the final stages of the interpreter exiting. The two conditions that trigger this are calling the flush() method on sys.stdout and sys.stderr. If you add a dummy flush() implementation, it no longer exits with 120:

>>> subprocess.call((sys.executable, "-c", """
... class NullWriter:
...     def write(self, s): pass
... import sys; sys.stderr = NullWriter()"""))
120
>>> subprocess.call((sys.executable, "-c", """
... class NullWriter:
...     def write(self, s): pass
...     def flush(self): pass
... import sys; sys.stderr = NullWriter()"""))
0

It does not seem to be explicitly documented what you can set sys.stderr to, but I always thought it is safest to use an io.TextIOBase implementation. I would suggest to derive your NullWriter class from TextIOBase; that way you get a default flush() implementation for free.

Other options are to use StringIO() if you are not expecting too much output, or open(os.devnull, "w"). See also Issue 28864 about adding a predefined class like NullWriter to Python.

Having said all that, perhaps it would be reasonable to tolerate a missing flush() method, and not treat this as an error.

Stepping back a bit, I also suggest restoring sys.stderr after the test. Otherwise, you risk missing an error message. Try contextlib.redirect_stderr().