This has already been discussed and rejected (issue 14403).  In practice the distinction between a failure and an error is not useful, and a comment in the test is IMO clearer than a no-op context manager: you can use a positive sentence instead of the reader having to understand a double negative ("Make sure there is an alert present; this will raise if not.")

The fact that there is no method to check that there is an alert sounds like a missing feature somewhere outside of Python?

Finally, if having this would in your group's opinion make your test code better, you can add it to your own TestCase subclass.  But our conclusion was that it doesn't belong in the stdlib.