This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: include __del__ in Generator ABC
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: gvanrossum Nosy List: gvanrossum, mmcewen-g, pje, rhettinger
Priority: normal Keywords: patch

Created on 2019-11-26 00:29 by mmcewen-g, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 17384 closed python-dev, 2019-11-26 00:33
Messages (5)
msg357467 - (view) Author: Matt McEwen (mmcewen-g) * Date: 2019-11-26 00:29
The Generator ABC in the standard library lets users define objects that follow the Generator specification given in PEP342, and which can be used in the place of builtin generator objects. 

This was originally added in issue 24018

The ABC enforces that the methods `send`, `throw` and `close` are implemented, as per the spec. It doesn't enforce that `__del__` is implemented. PEP342 specifies that `__del__` be a wrapper for `close`, such that when a generator object is garbage collected `close` is called.

For a user (like me) implementing such a generator object by inheriting the Generator ABC, the resulting objects do not have `close` called when they are garbage collected. Fixing this requires manually implementing a `__del__` that calls `close`. 

Ideally from the user perspective, inheriting from Generator should be enough to guarantee that the object behaves like a generator object, provided the abstract methods are correctly implemented. This could be easily achieved by implementing `__del__` concretely in the ABC as a wrapper for `close`:

def __del__(self):
    self.close()
msg357469 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-11-26 00:48
Guido should decide this one.  The PEP isn't entirely clear whether __del__ is a CPython implementation detail, nor is it clear whether the intent was for the isinstance() to insist on __del__ being present.  Also, at one time __del__ interfered with garbage collection, but that may have been fixed.
msg357472 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-26 01:21
The PEP does not specify collections.Generator at all, so this is not just a matter of interpreting the PEP.

The presence of a __del__ method can cause subtle behavior changes to the GC, so I worry that adding __del__ to that class now is going to break currently-working code.

Not having seen the OP's Generator subclass, I wonder if they even meant to subclass collections.Generator?  It's pretty esoteric to subclass Generator -- usually it's better to subclass Iterator or Iterable.
msg357476 - (view) Author: Matt McEwen (mmcewen-g) * Date: 2019-11-26 03:32
My interpretation of issue 24018 was that the Generator ABC was trying to follow the PEP as much as possible, so that users were able to produce a custom generator object and have it behave just like a builtin generator object.

I know that subclassing Generator is unusual; In my case that is a custom object that needs to be an iterator and also enter context managers internally. This seemed to me a very similar problem that the `close` machinery for generators was aimed to address, and subclassing generator worked really nicely, with one exception.

It took me some time to understand why the context managers weren't being exited correctly when the generator was interrupted, and it was because my `close` wasn't being called when the object fell out of scope, as it would have been if my object was a builtin generator object. Manually implementing __del__ fixed this.
msg357477 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2019-11-26 03:42
I wouldn't have use a Generator subclass for that. Let's not destabilize the Generator class.
History
Date User Action Args
2022-04-11 14:59:23adminsetgithub: 83092
2019-11-26 03:42:03gvanrossumsetstatus: open -> closed
resolution: wont fix
messages: + msg357477

stage: patch review -> resolved
2019-11-26 03:32:44mmcewen-gsetmessages: + msg357476
2019-11-26 01:21:02gvanrossumsetmessages: + msg357472
2019-11-26 00:48:26rhettingersetassignee: gvanrossum

messages: + msg357469
nosy: + gvanrossum, pje, rhettinger
2019-11-26 00:33:45python-devsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16866
2019-11-26 00:29:58mmcewen-gcreate