Message384014
Ok, so I now understand a bit more, and think it's not a bug! But explaining it involves some fairly deep thinking about generators.
I'll try to explain my reasoning.
Let's take a simple example:
---
def foo():
try:
yield
except:
print("ERROR")
for x in foo():
print(1)
---
It's convenient to think of it as python adding an implicit throw StopIteration() at the end of the generator function, I'll also rewrite the code to be roughly equivalent:
---
1. def foo():
2. try:
3. yield None
4. except:
5. print("ERROR")
6. #throw StopIteration():
7.
8. foo_gen = foo()
9. while True:
10. try:
11. x = next(foo_gen)
12. except StopIteration:
13. break
14. print(1)
15. del foo_gen
---
Now, if we step through how python runs the code starting at line 8.:
8. Create foo() <- this just creates a generator object
9. Enter while loop (not interesting for now)
10. try:
11. call next() on our generator to get the next value
2. try: <- we're running the generator until we get a yield now
3. yield None <- yield None to parent code
11. x = None <- assign yielded value to x
12. except StopIteration <- no exception raised so this doesn't get triggered
14. print(1) <- Print 1 to the output
9. Reached end of while loop, return to the start
10. try:
11. Re-enter the generator to get the next value, starting from where we left it..
4. except: <- there was no exception raised, so this doesn't get triggered
6. (implicit) throw StopIteration because generator finished
12. `except StopIteration` <- this is triggered because generator threw StopIteration
13. break <- break out of while loop
15. remove foo_gen variable, and clean up generator.
<- Deleting `foo_gen` causes `.close()` to be called on the generator which causes a GeneratorExit exception to be raised in the generator, BUT generator has already finished, so the GeneratorExit does nothing.
--
This is basically how the for-loop in your example works, and you can see that there's no generator exception, BUT if we change the print(1) to print(i) and try again:
8. Create foo() <- this just creates a generator object
9. Enter while loop (not interesting for now)
10. try:
11. call next() on our generator to get the next value
2. try: <- we're running the generator until we get a yield now
3. yield None <- yield None to parent code
11. x = None <- assign yielded value to x
12. except StopIteration <- no exception raised so this doesn't get triggered
*** CHANGED BIT ***
14. print(i) <- i doesn't exist, so throw NameError
14. The exception made us exit the current stack frame, so start cleaning up/deleting local variables
<- Deleting `foo_gen` causes `.close()` to be called on the generator which causes GeneratorExit() to be raised within the generator, but the generator is currently paused on line 3. so raise exception as-if we're currently running line 3:
4. except: <- this broad except catches the GeneratorExit exception because it appears to have happened on line 3.
5. print("ERROR") <- We only get here if the above steps happened.
---
So, if you don't let a generator naturally finish itself, but stop consuming the generator before it's raised its final StopIteration, then when the variable goes out-of-scope, a GeneratorExit will be raised at the point of the last yield that it ran.
If you then catch that GeneratorExit, and enter a new un-consumed loop (as in your `yield from foo()` line), then that line will also create the same situation again in a loop..
I understand that this used to "work" in previous python versions, but actually, having dug into things a lot more, I think the current behaviour is correct, and previous behaviors were not correct.
The "bug" here is in the example code that is catching GeneratorExit and then creating a new generator in the except:, rather than anything in Python |
|
Date |
User |
Action |
Args |
2020-12-29 16:43:35 | stestagg | set | recipients:
+ stestagg, xxm |
2020-12-29 16:43:35 | stestagg | set | messageid: <1609260215.52.0.149559397124.issue42762@roundup.psfhosted.org> |
2020-12-29 16:43:35 | stestagg | link | issue42762 messages |
2020-12-29 16:43:35 | stestagg | create | |
|