> I use inspect.signature for getting information about callables
> (third-party and first-party) in my type checker:
> https://github.com/quora/pyanalyze/blob/master/pyanalyze/arg_spec.py#L436. 
>  In that context, I'd much rather get string annotations that I can
> process myself later than get an exception if the annotations aren't
> valid at runtime.

This use case is what the eval_str parameter is for.  Since you're dealing specifically with type hints (as opposed to annotations generally), you don't really care whether you get strings or valid objects--you need to handle both.  So in Python 3.10+ you can--and should!--call inspect.get_annotations() and inspect.signature() with eval_str=False.  If you do that you won't have any trouble.


Since I keep getting new proposals on how to suppress eval() errors in inspect.signature(), I think I need to zoom out and talk about why I don't want to do it at all.  Forgive me, but this is long--I'm gonna start from first principles.

Changing inspect.signature() so it calls eval() on annotations is a change in 3.10.  And, due to the fact that it's possible (how likely? nobody seems to know) that there are malformed string annotations lurking in user code, this has the possibility of being a breaking change.

In order to handle this correctly, I think you need to start with a more fundamental question: are *stringized* annotations supposed to be a hidden implementation detail, and Python should present annotations as real values whether or not they were internally stringized?  Or: if the user adds "from __future__ import annotation" to their module, is this them saying they explicitly want their annotations as strings and they should always see them as strings?

(Again, this is specifically when the *language* turns your annotations into strings with the "from __future" import.  I think if the user enters annotations as strings, the language should preserve those annotations as strings, and that includes the library.)

It seems to me that the first answer is right.  PEP 563 talks a lot about "here's how you turn your stringized annotations back into objects".  In particular, it recommends calling typing.get_type_hints(), which AFAIK has always called eval() to turn string annotations back into objects.  It's worth noting here that, in both 3.9 and in the current Python trunk, typing.get_type_hints() doesn't catch exceptions.

Also, during the development of Python 3.10, during a time when stringized annotations had become the default behavior, inspect.signature() was changed to call typing.get_type_hints().  Presumably to have typing.get_type_hints() handle the tricky work of calling eval().

(I had problems with this specific approach.  "annotations" and "type hints" aren't the same thing, so having inspect.signature() e.g. wrap some annotations with Optional, and change None to NoneType, was always a mistake.  Also, typing.get_type_hints() was changed at this time to catch "Exception" and suppress *all* errors raised during the eval() call.  Happily both these changes have since been backed out.)

From that perspective, *not* having inspect.signature() turn stringized annotations back into strings from the very beginning was a bug.  And changing inspect.signature() in Python 3.10 so it calls eval() on string annotations is a bug fix.  So far folks seem to agree--the pushback I'm getting is regarding details of my approach, not on the idea of doing it at all.


Now we hit our second question.  It's possible that inspect.signature() can't eval() every stringized annotation back into a Python value, due to a malformed annotation expression that wasn't caught at compile-time.  This means inspect.signature() calls that worked in 3.9 could potentially start failing in 3.10.  How should we address it?

From the perspective of "string annotations are a hidden implementation detail, and users want to see real objects", I think the fact that it wasn't already failing was also a bug.  If your annotations are malformed, surely you want to know about it.  If you have a malformed annotation, and you don't turn on stringized annotations, you get an exception when the annotation expression is evaluated at module import time, in any Python version up to an including 3.10.  Stringized annotations delays this evaluation to when the annotations are examined--which means the exception for a malformed expression gets delayed too.  Which in turn means, from my perspective, the fact that inspect.signature() didn't raise on malformed annotation expressions was a bug.

I just don't agree that silently changing the failed eval()'d string annotation into something else is the right approach.  There have been a bunch of proposals along these lines, and while I appreciate the creative contributions and the attempts at problem-solving, I haven't liked any of them.  But it's not that I find that particular spelling awful, and if we could work together we'll find a spelling that I like.  I don't like the basic approach of silently suppressing these errors--I think inspect.signature() raising an exception is *already* the right behavior.  Again I'll quote from the Zen: "errors should never pass silently unless explicitly silenced", and, "special cases aren't special enough to break the rules".

(For the record, Guido's new proposal of "add a field to the Parameter object indicating the error" is the best proposal yet.  I don't want to do this at all, but if somehow it became simply unavoidable, ATM that seems like the best approach to take.)

I must admit I'm a little puzzled by the pushback I'm getting.  I thought the pushback would be "having inspect.signature() call eval() at all is a breaking change, it's too late to change it".  But the pushback has been all about coddling users with malformed annotations, by making the errors pass silently.  We're talking about user code that has errors, and these errors have been passing silently, potentially for years.  I regret the inconvenience that 3.10 will finally start raising an exception for these errors--and yet it still seems like an obviously better design than silently suppressing these errors.

(I also think that catching errors quickly goes down a rabbit hole.  Should inspect.signature() also suppress MemoryError? RecursionError? ImportError?)



Finally, there's an extant use case for code bases to deliberately provide malformed, un-evaluatable annotations.  This is due to structural problems endemic with large code bases (circular dependencies / circular imports).  Again I think the Zen's guidance here is right, and I think it's reasonable to expect code bases that deliberately break the rules to work around this change.  I realize that this bug fix in the standard library will inconvenience them--but that's why I made sure to have something like eval_str=False.  I have empathy for the bind these users find themselves in, and I want to ensure they have the tools they need to succeed.  But I think they need to work around bug fixes in the library, rather than preserve the old bug they were implicitly relying on.

And there's one more loose thread that I need to tie off here: what about users with objects with manually-entered string annotations that are deliberately not valid Python, who examine these objects with inspect.signature()?  I regret that this is collateral damage.  "from __future__ import annotations" doesn't give us any way to distinguish between "this annotation was automatically stringized by Python" and "this annotation is a string explicitly entered by the user".  My heuristic with the default behavior of inspect.get_annotations() and inspect.signature() (see eval_str=ONLY_IF_STRINGIZED) is an attempt to accommodate such usage.  But this heuristic won't do the right thing if the user manually stringizes all the annotations for an object.  In this case, this is legitimately permissible code, which will start failing in Python 3.10.  I apologize in advance to such users.  But my guess is this is exceedingly rare, and again such users can switch to eval_str=False at which point they'll be back in business.