What is happening here is that the __exit__ method gets passed the exception, and then instead of returning and allowing the exception to propagate and be printed, it raises SystemExit (via parse_args), which causes Python to immediately shut down, *before* __exit__ returns and the exception is propagated.

So yes, you'd have to do something specific for argparse (which raises a SystemExit exception if parsing fails) if you want to do something non-normal with SystemExit.  That is, you'll need to catch SystemExit.

This is really a duplicate of issue 9938.