classification
Title: typing.get_type_hints does not accept type annotations with leading whitespaces
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.10
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: gousaiyang, gvanrossum, jarryshaw, kj, levkivskyi
Priority: normal Keywords: patch

Created on 2021-04-20 13:03 by jarryshaw, last changed 2021-09-14 20:55 by gvanrossum. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 25484 closed jarryshaw, 2021-04-20 14:46
Messages (8)
msg391434 - (view) Author: Jarry Shaw (jarryshaw) * Date: 2021-04-20 13:03
`typing.get_type_hints` does not accept type annotations (in string capsulated form) with leading whitespaces.

```
>>> def foo(arg) -> ' str': ...
>>> typing.get_type_hints(foo)
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/typing.py", line 519, in __init__
    code = compile(arg, '<string>', 'eval')
  File "<string>", line 1
    str
IndentationError: unexpected indent

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/typing.py", line 1448, in get_type_hints
    value = ForwardRef(value)
  File "/usr/local/lib/python3.9/typing.py", line 521, in __init__
    raise SyntaxError(f"Forward reference must be an expression -- got {arg!r}")
SyntaxError: Forward reference must be an expression -- got ' str'
```

When elaborating on this issue, an inconsistency of hevaiour between ``eval`` function call and ``compile`` in ``'eval'`` mode was also noticed, where the former accepts leading whitespaces and the latter does not.

However, as discussed in https://bugs.python.org/issue41887 , I would then propose manually ``lstrip`` whitespaces from the type annotations:

```
519c519
<             code = compile(arg.lstrip(' \t'), '<string>', 'eval')
---
>             code = compile(arg, '<string>', 'eval')
```
msg391444 - (view) Author: Jarry Shaw (jarryshaw) * Date: 2021-04-20 15:23
> as discussed in https://bugs.python.org/issue41887

After some additional thoughts, I am thinking that changing the behaviour of ``compile`` in ``'eval'`` mode directly might be a better idea, for consistency all over the builtin functions. 

Especially that it is a bit complex to decide if ``compile`` is called in ``'eval'`` mode all over the code base which may need to patch with ``lstrip`` sanitisation.
msg391445 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-04-20 15:49
So the question is, whether anyone actually writes `x: ' str'`. Does the fix satisfy a real need?

If it does, why don't we change compile(x, ..., 'eval') as you suggested in your second comment?
msg391450 - (view) Author: Jarry Shaw (jarryshaw) * Date: 2021-04-20 16:20
To me, I discovered this issue because of a typo in my code. And apparently, I do not suggest people will write their type annotations with leading whitespaces purposely.

Here’s another thing though: aligning the behaviour of the two builtin functions is good for us users, since logically, ``compile`` is just one *execution* step away from ``eval`` to my understanding.
msg391451 - (view) Author: Saiyang Gou (gousaiyang) * Date: 2021-04-20 16:23
I think it might be a good idea to just strip leading spaces and tabs for `compile(x, ..., 'eval')` if we want consistent behavior. `compile` might be used in more locations in the whole Python source tree apart from `typing.get_type_hints`. Technically the only behavior difference between `eval(x)` and `compile(x, ..., 'eval')` (when `x` is a string) should be that the latter one does not execute the generated byte code.
msg394690 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-05-28 21:48
Also at this point it's too late to get this into 3.10 (given that we're still debating this it's clearly not a simple bugfix).
msg397682 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-07-17 03:12
The more I think about this, the less I like it. AFAIK static checkers like mypy don’t strip leading white space from forward references either. (If I’m wrong, please show evidence.)
msg401799 - (view) Author: Jarry Shaw (jarryshaw) * Date: 2021-09-14 20:47
Apparently static checkers like mypy doesn't rely on the `typing.get_type_hints` function to implement its type checking functions (as I had browsed through the code repo).

$ cat test.py
def foo(arg) -> ' str': ...
$ mypy test.py
Success: no issues found in 1 source file

If type checkers *think* this is acceptable, but the standard library doesn't, this can be some sort of inconsistency in my perspective.

As Saiyang Gou had suggested, I would rather prefer to change the behaviour of `compile` in `eval` mode to preserve the internal consistency of builtin functions and therefore to eliminate this *buggy* behaviour of `typing.get_type_hints`.
History
Date User Action Args
2021-09-14 20:55:00gvanrossumsetstatus: open -> closed
resolution: wont fix
stage: patch review -> resolved
2021-09-14 20:47:26jarryshawsetmessages: + msg401799
2021-07-17 03:12:30gvanrossumsetmessages: + msg397682
2021-05-28 21:48:02gvanrossumsetmessages: + msg394690
2021-04-20 16:23:10gousaiyangsetnosy: + gousaiyang
messages: + msg391451
2021-04-20 16:20:57jarryshawsetmessages: + msg391450
2021-04-20 15:49:40gvanrossumsetmessages: + msg391445
2021-04-20 15:23:59jarryshawsetmessages: + msg391444
2021-04-20 15:17:07kjsetnosy: + gvanrossum, levkivskyi, kj
2021-04-20 14:46:38jarryshawsetkeywords: + patch
stage: patch review
pull_requests: + pull_request24209
2021-04-20 13:03:19jarryshawcreate