There may be a (deliberate? :-) misunderstanding. When I wrote about "you" inspecting code by a "3rd party" I meant that as a symmetric relationship -- the "you" could be a library and from the library's POV the "3rd party" could be you (or me).

Either way, given that inspect.signature() collects many disparate aspects of the parameters of a function, it seems a bad design for it to raise an exception if any one of those aspects of any one of those parameters doesn't match an expectation (other than an expectation enforced by the parser+compiler themselves, like the constraint that positional-only parameters must precede other types of parameters).

I looked at the code, and there's a special value that Parameter().annotation is set if there's no annotation (Parameter.empty). Maybe you can follow that model and add another special value for errors during evaluation?