classification
Title: get_type_hints does not provide localns for classes
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, kj, larry, pbryan
Priority: normal Keywords: patch

Created on 2021-01-12 05:00 by pbryan, last changed 2021-04-29 23:48 by kj. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 24201 merged kj, 2021-01-12 15:22
PR 25632 merged kj, 2021-04-26 15:04
Messages (12)
msg384885 - (view) Author: Paul Bryan (pbryan) * Date: 2021-01-12 05:00
According to PEP 563:

> The get_type_hints() function automatically resolves the correct value of globalns for functions and classes. It also automatically provides the correct localns for classes.

This statement about providing correct localns for classes does not appear to be true.

Guido suggested this should be treated as a bug.
msg384887 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-12 05:12
Fidget-Spinner, are you interested in taking this?
msg384888 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-12 05:13
It's apparently a bug in all versions that support `from __future__ import annotations` (and only when that is used). Though perhaps we should only fix in in 3.10.
msg384925 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-01-12 12:33
> Fidget-Spinner, are you interested in taking this?

Sure thing! Please give me some time to look at it - I don't really use the runtime type validation stuff from typing (I usually defer that to 3rd party libraries), so I need to familiarize myself first.
msg390871 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-04-12 17:23
New changeset 852150ddfe68bc2696fc880175aeb855a0c16ae6 by Ken Jin in branch 'master':
bpo-42904: Fix get_type_hints for class local namespaces (GH-24201)
https://github.com/python/cpython/commit/852150ddfe68bc2696fc880175aeb855a0c16ae6
msg390872 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-04-12 17:24
Hi Ken Jin, you can close this issue now with your new permissions, right?
msg390945 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-04-13 09:48
Yup I can! BTW, I agree with keeping the change in 3.10 only. I suspect it _may_ be backwards-incompatible in small edge cases (re: the long comment on the PR about suspicions with local and global scope).
msg391955 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-04-26 17:31
New changeset 1b1f9852bda85c01ef124858f293e9c13e04ffce by Ken Jin in branch 'master':
bpo-42904: Change search order of typing.get_type_hints eval (#25632)
https://github.com/python/cpython/commit/1b1f9852bda85c01ef124858f293e9c13e04ffce
msg392317 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2021-04-29 14:30
kj: I just added support for default locals to inspect.get_annotation(), and I basically copied-and-pasted your dict(vars(base)) approach.

Is this "surprising, but required" behavior due specifically to this being a backwards-incompatible change? inspect.get_annotations() will be a new function in Python 3.10, which means I have no existing users to worry about. Does that mean I don't have the problem you're solving by reversing the order of namespace lookup, and I can just pass globals and locals in like normal?
msg392321 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-04-29 15:36
@larry
> Is this "surprising, but required" behavior due specifically to this being a backwards-incompatible change?

Yes. That's the main factor. I've since learnt that there's sadly more to it though :( (see below).

> Does that mean I don't have the problem you're solving by reversing the order of namespace lookup, and I can just pass globals and locals in like normal?

I think it depends on the ergonomics of the function you're trying to achieve. I admit I haven't been fully keeping up with the inspect.get_annotations issue (sorry!), but here's what I learnt from get_type_hints:

(Partly copied over from PR 25632)
Example:

from typing import TypeVar, Generic
T = TypeVar('T')

class A(Generic[T]):
    __annotations__ = dict(
        some_b='B'
    )


class B(Generic[T]):
    class A(Generic[T]):
        pass
    __annotations__ = dict(
        my_inner_a1='B.A',
        my_inner_a2=A,
        my_outer_a='A'  # unless somebody calls get_type_hints with localns=B.__dict__
    )

>>> get_type_hints(B)

Currently (globalns then localns):
{'my_inner_a1': <class '__main__.B.A'>, 'my_inner_a2': <class '__main__.B.A'>, 'my_outer_a': <class '__main__.A'>}

Swapped (localns then globalns):
{'my_inner_a1': <class '__main__.B.A'>, 'my_inner_a2': <class '__main__.B.A'>, 'my_outer_a': <class '__main__.B.A'>}

I realized that looking into globalns then localns is a necessary evil: doing the converse (looking into localns first then globalns) would mean there is no way to point to the shadowed global `A`: it would always point to the local `B.A`. Unless of course you pass in localns=module.__dict__ or localns=globals().

Ultimately I think it's a sacrifice of ergonomics either way; looking into localns then globalns will require passing in the module's __dict__ to refer to a global being shadowed, while the converse (globalns then localns) introduces surprising eval behavior. Both are kind of tacky, but globalns then localns is slightly less so IMO. If the user wants to specify the inner class, they can just annotate 'B.A', if they want the outer, it's 'A'. But the localns then globalns approach will always point to `B.A`, the only way to access the shadowed global `A` is to pass in the strange looking argument localns=module.__dict__.

Phew, my head's spinning with localns and globalns now. Thanks for reading! I think it's your call. I'm inexperienced with elegant function design :P.
msg392351 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2021-04-29 20:48
Thank you for your in-depth and thoughtful reply!

I think that APIs should behave in a predictable way.  So, for my use case, I tell the user that I'm passing "globals" and "locals" into eval()--and I think I'd have to have a *very* compelling reason to swap them.  Since I don't have the backwards-compatibility problem you do, I think I should keep it simple and predictable and not swap them.

In reference to your example, I think it's natural enough that the A defined inside B eclipses the module-level A.  That's what the user would *expect* to happen.  If the user really wants to reference the module-level A, they have lots of options:

  * rename one or the other A to something else
  * establish an alias at module scope, and use that
  * explicitly say globals()['A']

So I'm not worried about it.
msg392364 - (view) Author: Ken Jin (kj) * (Python committer) Date: 2021-04-29 23:48
Hmm I think you're right - the normal behavior in Python is for local
variables to shadow global variables, and as a user I expect the same for
eval in inspect.get_annotation.
History
Date User Action Args
2021-04-29 23:48:45kjsetmessages: + msg392364
2021-04-29 20:48:07larrysetmessages: + msg392351
2021-04-29 15:36:15kjsetmessages: + msg392321
2021-04-29 14:30:32larrysetmessages: + msg392317
2021-04-26 17:31:34gvanrossumsetmessages: + msg391955
2021-04-26 15:04:35kjsetpull_requests: + pull_request24327
2021-04-13 09:48:40kjsetstatus: open -> closed
resolution: fixed
messages: + msg390945

stage: patch review -> resolved
2021-04-12 17:24:18gvanrossumsetmessages: + msg390872
2021-04-12 17:23:20gvanrossumsetmessages: + msg390871
2021-01-12 15:22:13kjsetkeywords: + patch
stage: patch review
pull_requests: + pull_request23026
2021-01-12 12:33:43kjsetnosy: + kj
messages: + msg384925
2021-01-12 05:13:26gvanrossumsetmessages: + msg384888
2021-01-12 05:12:27gvanrossumsetmessages: + msg384887
2021-01-12 05:00:58pbryansetnosy: + larry
2021-01-12 05:00:15pbryancreate