Issue42725
Created on 2020-12-23 13:28 by BTaskaya, last changed 2021-01-18 20:09 by gvanrossum.
Messages (21) | |||
---|---|---|---|
msg383647 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2020-12-23 13:28 | |
Since the annotations are processed just like all other expressions in the symbol table, the generated entries for functions etc. This would result with def foo(): for number in range(5): foo: (yield number) return number foo() returning a generator / coroutine (depending on yield/yield from/await usage). Is this something we want to keep or maybe tweak the symbol table generator to not to handle annotations (since there are also more subtle issues regarding analysis of cell / free vars). |
|||
msg383655 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2020-12-23 19:35 | |
Ouch. I think we should generate a SyntaxError if yield [from] is used in an annotation. |
|||
msg383659 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2020-12-23 20:41 | |
This is another side effect of processing annotations (at the symbol table construction stage) (and I would assume there are a few more cases like this); def foo(): outer_var = 1 def bar(): inner_var: outer_var = T return bar inner = foo() print(inner.__closure__) In theory, there shouldn't be any cells / references to the variables from outer scope, but since we process the entry for the annotation and record `outer_var` as a free var it is listed here. |
|||
msg383660 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2020-12-23 20:45 | |
The difference is that that just causes a slight inefficiency, while processing 'yield' makes the function a generator. So I don't feel as strongly about that. |
|||
msg383661 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2020-12-23 21:09 | |
> So I don't feel as strongly about that. Just to note, since I believe the solution for all this might be the same (not processing annotations at all, since they will be compiled to strings in the later stage). If we go down on that route, it will be simpler but we won't be able to raise SyntaxError's for "a: (yield)" / "b: (await/yield from x)". By the way, this is what happens if you try to use get_type_hints on a function where an argument is (yield): >>> typing.get_type_hints(a) Traceback (most recent call last): File "/usr/local/lib/python3.10/typing.py", line 537, in __init__ code = compile(arg, '<string>', 'eval') File "<string>", line 1 SyntaxError: 'yield' outside function During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.10/typing.py", line 1490, in get_type_hints value = ForwardRef(value) File "/usr/local/lib/python3.10/typing.py", line 539, in __init__ raise SyntaxError(f"Forward reference must be an expression -- got {arg!r}") SyntaxError: Forward reference must be an expression -- got '(yield)' |
|||
msg383662 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2020-12-23 21:19 | |
If we simply ignored yield in the annotation, wouldn't we have the problem that def f(a: (yield)): pass silently changes from being a generator (in 3.9) to not being a generator (in 3.10)? That would be bad. I'd rather make this an error still. (But for nonlocals, not processing sounds fine.) |
|||
msg383686 - (view) | Author: Lysandros Nikolaou (lys.nikolaou) * ![]() |
Date: 2020-12-24 12:44 | |
I concur with Guido. I feel that making this an error is the best alternative we have. |
|||
msg383693 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2020-12-24 16:59 | |
Okay, let's do it. It should probably a post-parse check (before we invoke the bytecode compiler but after we have the AST in hand). |
|||
msg383730 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2020-12-25 09:33 | |
I have a patch ready to go, but I'd prefer to block this issue until issue 42737 is resolved/decided. |
|||
msg383812 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2020-12-26 17:46 | |
One thing to note here, currently Pablo and I are trying to bring annotation unparsing from the compiler to the parser in issue 41967. If we do so, the annotations won't cause any side effects on the symbol table generation. |
|||
msg384483 - (view) | Author: Mark Shannon (Mark.Shannon) * ![]() |
Date: 2021-01-06 10:12 | |
I've also opened #42837 which is about fixing the symbol table, so that it is correct w.r.t. to current behavior. I'd like to fix it ASAP as the compiler should be able to rely on the symbol table being correct. Of course, once we have decided what the behavior should be, then it may need to be updated again. I'm inclined to agree that 'yield' in an annotation should probably be a syntax error, but I haven't put much thought into to. |
|||
msg384508 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2021-01-06 15:38 | |
Yield in an annotation should be a syntax error. |
|||
msg384580 - (view) | Author: Mark Shannon (Mark.Shannon) * ![]() |
Date: 2021-01-07 11:42 | |
What's the process for making a decision on whether to make 'yield' in an annotation a syntax error? As a language change it should have a PEP, IMO. The PEP will be short, and shouldn't need a long-winded acceptance process. I just think that a PEP is more visible to the community than the bug tracker. I'm happy to write the PEP. |
|||
msg384589 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2021-01-07 14:39 | |
Should not "await" and "async for" (in comprehesions) and ":=" be forbidden too? |
|||
msg384602 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2021-01-07 21:07 | |
I wouldn’t have thought you’d need a PEP for this but if you want to write one that sounds like the right thing. Re: async and walrus: I think those are different, their presence doesn’t affect the meaning of the function like yield. We can’t hope to prevent side effects syntactically. |
|||
msg384617 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2021-01-07 22:44 | |
Does not walrus affect the meaning of variable? And await affects the meaning of generator expression. |
|||
msg384638 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2021-01-08 05:59 | |
Oh, you’re right about walrus. And I don’t actually understand async generator expressions. This suggests that we definitely need a PEP. :-) |
|||
msg385203 - (view) | Author: Mark Shannon (Mark.Shannon) * ![]() |
Date: 2021-01-18 15:57 | |
Draft PEP here https://github.com/markshannon/peps/blob/annotation-syntax-errors/pep-9999.rst Guido, would you like to be co-author as it was your idea to make these things a syntax error? |
|||
msg385210 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2021-01-18 18:07 | |
It'll probably be quicker if you leave me out of the loop (feel free to quote me though). I am currently quite overwhelmed with PEP-sized discussions. I expect that the SC can rule on this quickly (just use their tracker to send them the PEP). |
|||
msg385217 - (view) | Author: Batuhan Taskaya (BTaskaya) * ![]() |
Date: 2021-01-18 18:38 | |
> The use of nonlocal variables in an annotation will be a syntax error in Python 3.10 What is the reasoning for forbidding nonlocal variables (https://bugs.python.org/msg383659), can't we just treat them like regular variables and leave the value to whom needed to deal with resolving the scope? Also, you should preferably clarify other cases regarding the symbol table interaction of annotations. For example this case: Here is one; class T: def foo(self): a: super().bar() = x print(T.foo.__closure__) And if we are on the road to writing a PEP, maybe we should somehow squeeze issue 42737 somewhere, since the annotations for complex targets are still evaluated and this makes everything a bit problematic and inconsistent. If you need collaboration on the PEP, just let me know (isidentical@gmail.com) (I have a patch ready to go for the symbol table to both make annotations ineffective and forbid this stuff but was waiting for issue 42737) |
|||
msg385223 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2021-01-18 20:09 | |
I'm not keen on prohibiting nonlocal variable reference in annotations, since I can imagine uses for this (e.g. in tests). I'm not too worried about keeping the cell alive "forever", since with Larry's PEP 649 their lifetime would end when the object containing the annotation ends. Someone should look into how Batuhan's super()-in-annotation example would behave under Larry's PEP 649. (Using super() in the magical function Larry proposes to generate might not work anyway, since super()'s own magical powers don't work in a nested function.) Re: issue42737, not sure what to do for that, since neither PEP 563 nor PEP 649 gives a way to access those annotations. Then again, that's also true for annotations on local variables. So maybe we can treat them the same? Type checkers won't care, and runtime uses of annotations (whether as type hints or otherwise) won't have access...... |
History | |||
---|---|---|---|
Date | User | Action | Args |
2021-01-18 20:09:59 | gvanrossum | set | messages: + msg385223 |
2021-01-18 18:38:18 | BTaskaya | set | messages: + msg385217 |
2021-01-18 18:07:38 | gvanrossum | set | messages: + msg385210 |
2021-01-18 15:57:23 | Mark.Shannon | set | messages: + msg385203 |
2021-01-08 05:59:05 | gvanrossum | set | messages: + msg384638 |
2021-01-07 22:44:26 | serhiy.storchaka | set | messages: + msg384617 |
2021-01-07 21:07:44 | gvanrossum | set | messages: + msg384602 |
2021-01-07 14:39:28 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg384589 |
2021-01-07 11:42:48 | Mark.Shannon | set | nosy:
+ larry messages: + msg384580 |
2021-01-06 15:38:20 | gvanrossum | set | messages: + msg384508 |
2021-01-06 10:12:29 | Mark.Shannon | set | nosy:
+ Mark.Shannon messages: + msg384483 |
2021-01-06 10:05:22 | Mark.Shannon | unlink | issue42837 superseder |
2021-01-06 09:59:23 | serhiy.storchaka | link | issue42837 superseder |
2020-12-26 17:46:24 | BTaskaya | set | nosy:
+ pablogsal messages: + msg383812 |
2020-12-25 09:33:24 | BTaskaya | set | messages: + msg383730 |
2020-12-24 16:59:15 | gvanrossum | set | messages: + msg383693 |
2020-12-24 12:44:42 | lys.nikolaou | set | messages: + msg383686 |
2020-12-23 21:19:59 | gvanrossum | set | messages: + msg383662 |
2020-12-23 21:09:33 | BTaskaya | set | messages: + msg383661 |
2020-12-23 20:45:23 | gvanrossum | set | messages: + msg383660 |
2020-12-23 20:41:34 | BTaskaya | set | messages: + msg383659 |
2020-12-23 19:35:14 | gvanrossum | set | nosy:
+ lys.nikolaou |
2020-12-23 19:35:04 | gvanrossum | set | messages: + msg383655 |
2020-12-23 13:28:51 | BTaskaya | create |