msg326148 - (view) |
Author: David Hagen (drhagen) |
Date: 2018-09-23 10:45 |
The new postponed annotations have an unexpected interaction with dataclasses. Namely, you cannot get the type hints of any of the data classes methods.
For example, I have some code that inspects the type parameters of a class's `__init__` method. (The real use case is to provide a default serializer for the class, but that is not important here.)
```
from dataclasses import dataclass
from typing import get_type_hints
class Foo:
pass
@dataclass
class Bar:
foo: Foo
print(get_type_hints(Bar.__init__))
```
In Python 3.6 and 3.7, this does what is expected; it prints `{'foo': <class '__main__.Foo'>, 'return': <class 'NoneType'>}`.
However, if in Python 3.7, I add `from __future__ import annotations`, then this fails with an error:
```
NameError: name 'Foo' is not defined
```
I know why this is happening. The `__init__` method is defined in the `dataclasses` module which does not have the `Foo` object in its environment, and the `Foo` annotation is being passed to `dataclass` and attached to `__init__` as the string `"Foo"` rather than as the original object `Foo`, but `get_type_hints` for the new annotations only does a name lookup in the module where `__init__` is defined not where the annotation is defined.
I know that the use of lambdas to implement PEP 563 was rejected for performance reasons. I could be wrong, but I think this was motivated by variable annotations because the lambda would have to be constructed each time the function body ran. I was wondering if I could motivate storing the annotations as lambdas in class bodies and function signatures, in which the environment is already being captured and is code that usually only runs once.
Original mailing list discussion: https://mail.python.org/pipermail/python-dev/2018-September/155289.html
|
msg326164 - (view) |
Author: Eric V. Smith (eric.smith) *  |
Date: 2018-09-23 16:35 |
[Adding to nosy people who were on the original email]
Copying (part of) my response from the email thread:
These work:
print(get_type_hints(Bar.__init__, globals()))
print(get_type_hints(Bar.__init__, Bar.__module__))
But I agree that maybe doing something with dataclasses to address this would be good. Especially as the first one requires being in the same module as Foo.
See this for Yury's self-described "hack-ish fix we can use" until we do something better:
https://gist.github.com/1st1/37fdd3cc84cd65b9af3471b935b722df
|
msg326170 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2018-09-23 18:31 |
> See this for Yury's self-described "hack-ish fix we can use" until we do something better:
Actually, I think I found a better solution that doesn't require any changes to anything besides dataclasses.
Currently, dataclasses uses 'exec()' function to dynamically create methods like '__init__'. The generated code for '__init__' needs to access MISSING and _HAS_DEFAULT_FACTORY constants from the dataclasses module. To do that, we compile the code with 'exec()' with globals set to a dict with {MISSING, _HAS_DEFAULT_FACTORY} keys in it. This does the trick, but '__init__.__globals__' ends up pointing to that custom dict, instead of pointing to the module's dict.
The other way around is to use a closure around __init__ to inject MISSING and _HAS_DEFAULT_FACTORY values *and* to compile the code in a proper __dict__ of the module the dataclass was defined in. Please take a look at the PR.
|
msg326171 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2018-09-23 18:33 |
And FWIF I don't think we need to use lambdas for annotations to solve issues like this one.
|
msg326266 - (view) |
Author: Eric V. Smith (eric.smith) *  |
Date: 2018-09-24 16:45 |
Ned: I'm marking this as a release blocker because I'd like to get it in 3.7.1. If I can't do that in the next 5 hours or so, I'll remove the release blocker tag.
|
msg326307 - (view) |
Author: Eric V. Smith (eric.smith) *  |
Date: 2018-09-25 01:07 |
Unfortunately, I'm not going to be able to give this the attention it deserves before the 3.7.1 cutoff. The changes are sufficiently tricky that I want to make sure I think through the issue in the appropriate detail.
Ned: I've made it not a release blocker. Please remove yourself as nosy if you don't care about this issue. I don't plan on making it a release blocker again, I just did that because it was so near the deadline.
Yury: Sorry for not being able to get delve in to it sufficiently. Thanks for your work on it.
I should be able to review this in the next few days, so we can get this in to 3.7.2.
|
msg326308 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2018-09-25 01:49 |
NP, Eric, take your time. I agree that the PR isn't simple and needs a very careful review.
|
msg326314 - (view) |
Author: Gregory P. Smith (gregory.p.smith) *  |
Date: 2018-09-25 04:02 |
fwiw, agreed that this should wait for 3.7.2.
|
msg327059 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2018-10-04 15:55 |
Yury, thanks for your patch. I'll review it soon.
Please note that postponed annotations only reveal a problem that we already had if anybody used a string forward reference:
>>> from dataclasses import dataclass
>>> from typing import get_type_hints
>>> class C:
... pass
...
>>> @dataclass
... class D:
... c: C
...
>>> @dataclass
... class E:
... c: "C"
...
>>> get_type_hints(C.__init__)
{}
>>> get_type_hints(D.__init__)
{'c': <class '__main__.C'>, 'return': <class 'NoneType'>}
>>> get_type_hints(E.__init__)
Traceback (most recent call last):
...
NameError: name 'C' is not defined
|
msg327062 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2018-10-04 16:06 |
> Please note that postponed annotations only reveal a problem that we already had if anybody used a string forward reference:
Yeah, makes sense. It's cool that the PR fixes string forward references as well.
|
msg337014 - (view) |
Author: Vlad Shcherbina (vlad) * |
Date: 2019-03-02 16:20 |
Any chance this could get into 3.7.3?
|
msg337237 - (view) |
Author: Eric V. Smith (eric.smith) *  |
Date: 2019-03-05 19:15 |
I'm finally getting time to look at this. I'll see what I can do.
|
msg355258 - (view) |
Author: David Hagen (drhagen) |
Date: 2019-10-23 19:27 |
This PR has been sitting for a while. Any chance we can bring it over the finish line?
|
msg355268 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2019-10-23 21:53 |
FWIW we discussed my patch with Eric at PyCon 2019 and I think he had no issues with it. Eric, can I merge it? Should we backport to 3.8?
|
msg355269 - (view) |
Author: Eric V. Smith (eric.smith) *  |
Date: 2019-10-23 22:27 |
Yury: I'm okay with merging. If I recall, you were going to add a comment or two about the approach (a closure, if I remember correctly).
I think we should backport to 3.8 and 3.7: it's a bug.
|
msg355270 - (view) |
Author: Yury Selivanov (yselivanov) *  |
Date: 2019-10-23 22:29 |
> Yury: I'm okay with merging. If I recall, you were going to add a comment or two about the approach (a closure, if I remember correctly).
Sure, I'll rebase and add a comment in a couple of days. I'll elevate the status so that I don't forget before 3.8.1
|
msg357987 - (view) |
Author: Ned Deily (ned.deily) *  |
Date: 2019-12-07 20:39 |
> I'll elevate the status so that I don't forget before 3.8.1.
Ping, 3.8.1 cutoff is coming up very soon.
|
msg358106 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2019-12-09 14:54 |
New changeset d219cc4180e7589807ebbef7421879f095e72a98 by Łukasz Langa (Yury Selivanov) in branch 'master':
bpo-34776: Fix dataclasses to support __future__ "annotations" mode (#9518)
https://github.com/python/cpython/commit/d219cc4180e7589807ebbef7421879f095e72a98
|
msg358112 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2019-12-09 16:07 |
New changeset 0d57db27f2c563b6433a220b646b50bdeff8a1f2 by Łukasz Langa (Miss Islington (bot)) in branch '3.8':
bpo-34776: Fix dataclasses to support __future__ "annotations" mode (GH-9518) (#17531)
https://github.com/python/cpython/commit/0d57db27f2c563b6433a220b646b50bdeff8a1f2
|
msg358113 - (view) |
Author: Łukasz Langa (lukasz.langa) *  |
Date: 2019-12-09 16:07 |
New changeset 66d7a5d58a88bce312bc4668f2cc54c9488c5bd8 by Łukasz Langa (Miss Islington (bot)) in branch '3.7':
bpo-34776: Fix dataclasses to support __future__ "annotations" mode (GH-9518) (#17532)
https://github.com/python/cpython/commit/66d7a5d58a88bce312bc4668f2cc54c9488c5bd8
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:06 | admin | set | github: 78957 |
2019-12-09 16:41:55 | lukasz.langa | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
2019-12-09 16:07:58 | lukasz.langa | set | messages:
+ msg358113 |
2019-12-09 16:07:55 | lukasz.langa | set | messages:
+ msg358112 |
2019-12-09 14:55:00 | miss-islington | set | pull_requests:
+ pull_request17010 |
2019-12-09 14:54:52 | miss-islington | set | pull_requests:
+ pull_request17009 |
2019-12-09 14:54:27 | lukasz.langa | set | messages:
+ msg358106 |
2019-12-07 20:39:18 | ned.deily | set | messages:
+ msg357987 |
2019-10-23 22:29:27 | yselivanov | set | priority: deferred blocker assignee: eric.smith -> yselivanov messages:
+ msg355270
|
2019-10-23 22:27:46 | eric.smith | set | messages:
+ msg355269 |
2019-10-23 21:53:11 | yselivanov | set | messages:
+ msg355268 |
2019-10-23 19:44:16 | gregory.p.smith | set | versions:
+ Python 3.9 |
2019-10-23 19:27:43 | drhagen | set | messages:
+ msg355258 |
2019-03-05 19:15:49 | eric.smith | set | messages:
+ msg337237 |
2019-03-02 16:20:28 | vlad | set | nosy:
+ vlad messages:
+ msg337014
|
2018-10-04 16:06:57 | yselivanov | set | messages:
+ msg327062 |
2018-10-04 15:55:03 | lukasz.langa | set | messages:
+ msg327059 |
2018-09-28 06:27:41 | scoder | set | nosy:
+ scoder
|
2018-09-27 17:06:43 | methane | set | nosy:
+ methane
|
2018-09-27 16:47:02 | levkivskyi | set | nosy:
+ levkivskyi
|
2018-09-25 04:02:43 | gregory.p.smith | set | messages:
+ msg326314 |
2018-09-25 04:01:50 | gregory.p.smith | set | nosy:
+ gregory.p.smith
versions:
+ Python 3.8 |
2018-09-25 01:49:35 | yselivanov | set | messages:
+ msg326308 |
2018-09-25 01:07:46 | eric.smith | set | priority: release blocker -> (no value)
messages:
+ msg326307 |
2018-09-24 16:45:53 | eric.smith | set | priority: normal -> release blocker
nosy:
+ ned.deily messages:
+ msg326266
assignee: eric.smith |
2018-09-23 18:33:27 | yselivanov | set | messages:
+ msg326171 |
2018-09-23 18:31:42 | yselivanov | set | messages:
+ msg326170 |
2018-09-23 18:25:32 | yselivanov | set | keywords:
+ patch stage: patch review pull_requests:
+ pull_request8923 |
2018-09-23 16:35:44 | eric.smith | set | nosy:
+ ivan, eric.smith, gvanrossum, lukasz.langa, yselivanov messages:
+ msg326164
|
2018-09-23 10:45:29 | drhagen | create | |