> Are you happy for me to write the test?

I had started working on that, but sure, be my guest. :)

You seem to be on the right track.  I didn't find precedent nearby either.  We basically want a C version of unittest's assertRaisesRegex() (but it can be a straight string match).  CHECK_INVALID comes close to that.

I would suggest defining a helper function that accepts an exception type and message text, and that clears the current error if it matches.  If the error doesn't match, you can restore the existing one with PyErr_Restore(), or else call raiseTestError() if there isn't one (similar to CHECK_INVALID).

I was thinking of putting the test right after the call to TESTNAME() in test_long_api(), but maybe you know a better location:

http://hg.python.org/cpython/file/default/Modules/_testcapimodule.c#l313

I'll upload what I had for the pure Python failing test since that portion was finished.