classification
Title: get_type_hints fails if there are un-annotated fields in a dataclass
Type: behavior Stage:
Components: Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: arne, eric.smith, levkivskyi, serhiy.storchaka
Priority: normal Keywords:

Created on 2019-08-26 06:29 by arne, last changed 2020-03-08 13:51 by serhiy.storchaka.

Messages (5)
msg350488 - (view) Author: Arne Recknagel (arne) Date: 2019-08-26 06:29
When declaring a dataclass with make_dataclass, it is valid to omit type information for fields. __annotations__ understands it and just adds typing.Any, but typing.get_type_hints fails with a cryptic error message:

>>> import dataclasses
>>> import typing
>>> A = dataclasses.make_dataclass('A', ['a_var'])
>>> A.__annotations__
{'a_var': 'typing.Any'}
>>> typing.get_type_hints(A)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/user/venvs/python_3.7/lib/python3.7/typing.py", line 973, in get_type_hints
    value = _eval_type(value, base_globals, localns)
  File "/user/venvs/python_3.7/lib/python3.7/typing.py", line 260, in _eval_type
    return t._evaluate(globalns, localns)
  File "/user/venvs/python_3.7/lib/python3.7/typing.py", line 464, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'typing' is not defined


Adding typing.Any explicitly is an obvious workaround:

>>> B = dataclasses.make_dataclass('B', [('a_var', typing.Any)])
>>> typing.get_type_hints(B)
{'a_var': typing.Any}

There is already a bug filed regarding datalcasses and get_type_hints which might be related: https://bugs.python.org/issue34776
msg350973 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2019-09-02 09:13
It looks like https://github.com/python/cpython/pull/9518 will fix also this one.
msg355559 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-10-28 16:16
I'm not sure what can be done with this. The problem is that the decorator doesn't know what's in the caller's namespace. The type being added is "typing.Any". If the caller doesn't import typing, then get_type_hints will fail (as demonstrated here).

The only thing I can think of is using a type that's in builtins. "object" springs to mine, but of course that's semantically incorrect.

Or, maybe I could use "dataclasses.sys.modules['typing'].Any". I don't currently import sys (I don't think), but this should be a cheap import. Then if typing.get_type_hints() is called, we know typing will have already been importing.

But what if "dataclasses" isn't in the caller's namespace? I guess if I could find some way to navigate to sys.modules from __builtins__, that would largely work, absent playing games with builtins.
msg356829 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2019-11-17 20:28
> I'm not sure what can be done with this. The problem is that the decorator doesn't know what's in the caller's namespace. The type being added is "typing.Any". If the caller doesn't import typing, then get_type_hints will fail (as demonstrated here).

IIUC the main problem is that get_type_hints() fails even if typing is imported. I would expect this to work (just repeating the original example in a more compact form):

import dataclasses
import typing
A = dataclasses.make_dataclass('A', ['a_var'])
typing.get_type_hints(A)  # This currently crashes

Interestingly, if I use a very similar call that it works:

>>> typing.get_type_hints(A, globalns=globals())
{'a_var': typing.Any}

So the core of the issue is that the globals are identified incorrectly, and indeed if I look at the generated class it looks wrong:

>>> A.__module__
'types'  # Should be '__main__'

I think we should fix the ``__module__`` attribute of the dynamically generated dataclasses (for example the way it is done for named tuples).

Btw, https://github.com/python/cpython/pull/14166 may potentially fix the ``__module__`` attribute here too.
msg363662 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-03-08 13:51
PR 14166 does not fix this issue.
History
Date User Action Args
2020-03-08 13:51:23serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg363662
2019-11-17 20:28:55levkivskyisetmessages: + msg356829
2019-10-28 16:16:57eric.smithsetmessages: + msg355559
2019-09-02 09:13:00levkivskyisetnosy: + levkivskyi
messages: + msg350973
2019-08-26 08:34:57eric.smithsetassignee: eric.smith
2019-08-26 06:29:12arnecreate