➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2015-01-13 07:35 by sjdrake, last changed 2022-04-11 14:58 by admin. This issue is now closed.
| Messages (4) | |||
|---|---|---|---|
| msg233903 - (view) | Author: Stephen Drake (sjdrake) * | Date: 2015-01-13 07:35 | |
If a generator has its close() method called before any items are requested from it, a finally block in the generator function will not be executed.
I encountered this when wrapping an open file to alter the result of iterating over it. Using a generator function with a try/finally block seemed like a simple way of acheiving this. Here's an example that logs each line as it's read:
def logged_lines(f):
try:
for line in f:
logging.warning(line.strip())
yield line
finally:
logging.warning('closing')
f.close()
If the generator is created and closed immediately, the underlying file-like object is left open:
>>> f = urlopen('https://docs.python.org/')
>>> lines = logged_lines(f)
>>> lines.close()
>>> f.closed
False
But once the first item is requested from the generator, close() will trigger cleanup:
>>> lines = logged_lines(f)
>>> next(lines)
WARNING:root:<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n'
>>> lines.close()
WARNING:root:closing
>>> f.closed
True
Having read the documentation for yield expressions, I don't believe this behaviour to be non-conformant - but it still seems like a bit of a gotcha to me. Should this usage be warned against?
|
|||
| msg241794 - (view) | Author: Marco Paolini (mpaolini) * | Date: 2015-04-22 10:17 | |
I think there is an issue in the way you designed your cleanup logic. So I think this issue is invalid.
Usually, the code (funcion, class, ...) that *opens* the file should also be resposible of closing it.
option 1) the caller opens and closes the file and wrapping the logged lines in a try/finally
def logged_lines(f):
try:
for line in f:
logging.warning(line.strip())
yield line
finally:
logging.warning('closing')
f = open('yyy', 'r')
try:
for l in logged_lines(f):
print(l)
finally:
f.close()
option 2) the funcion opens and closes the file
def logged_lines(fname):
f = open('yyy', 'r')
try:
for line in f:
logging.warning(line.strip())
yield line
finally:
logging.warning('closing')
f.close()
for l in logged_lines('yyy'):
print(l)
|
|||
| msg241833 - (view) | Author: Antoine Pitrou (pitrou) *  |
Date: 2015-04-22 23:11 | |
This looks logical to me. The "finally" block is only entered if the "try" block is ever entered, but if you don't consume anything in the generator then the generator's code is never actually executed. |
|||
| msg241911 - (view) | Author: Stephen Drake (sjdrake) * | Date: 2015-04-24 05:17 | |
Ok, I can accept that. I think my mistake was to assume that because a generator has a close() method, I could treat it as a lightweight wrapper for another closeable object.
But it's better to regard a generator function that wraps an iterable as something more akin to map() or filter(), and use a class if it's necessary to wrap a file such that close() is passed through.
I happened to take a fresh look at this just the other day and it also occurred to me that the kind of composition I was trying to do can work if it's generators all the way down:
def open_lines(name, mode='rt', buffering=-1):
with open(name, mode, buffering) as f:
for line in f:
yield line
def logged_lines(f):
try:
for line in f:
logging.warning(line.strip())
yield line
finally:
f.close()
lines = open_lines('yyy', 'r')
if verbose:
lines = logged_lines(lines)
try:
for line in lines:
print(line)
finally:
lines.close()
So a generator can transparently wrap a plain iterable or another generator, but not closeable objects in general. There's nothing really wrong with that, so I'm happy for this issue to be closed as invalid.
|
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:11 | admin | set | github: 67416 |
| 2015-05-14 23:22:23 | martin.panter | set | status: open -> closed resolution: not a bug stage: resolved |
| 2015-04-24 05:17:33 | sjdrake | set | messages: + msg241911 |
| 2015-04-22 23:11:07 | pitrou | set | nosy:
+ pitrou messages: + msg241833 |
| 2015-04-22 10:17:12 | mpaolini | set | nosy:
+ mpaolini messages: + msg241794 |
| 2015-04-22 04:10:12 | martin.panter | set | nosy:
+ martin.panter |
| 2015-01-13 13:12:06 | r.david.murray | set | nosy:
+ r.david.murray |
| 2015-01-13 07:35:46 | sjdrake | create | |