This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ncoghlan
Recipients martin.panter, ncoghlan, njs, vstinner, yselivanov
Date 2016-11-14.11:37:44
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1479123464.82.0.469895350315.issue28629@psf.upfronthosting.co.za>
In-reply-to
Content
Martin raises an interesting point - the cases where ResourceWarning is likely to be appropriate are going to be those where the frame has been suspended inside a with statement or a try/finally statement.

However, we don't currently track those at runtime or at compile time in any way - they're implicit in the state of the frame stack.

That doesn't mean we *couldn't* track them though, and I think co_lnotab provides a hint as to how we could do it with minimal runtime overhead: add a new co_cleanup data table to code objects that provides efficiently packed storage for a sequence of "setup/cleanup" bytecode offset 2-tuples.

All code objects would gain an additional pointer, and those that used with and try/finally would get fractionally larger than that, but frame objects would remain the same size they are today.

Glossing over the complexities of large bytecode offsets (ala co_lnotab), consider:

>>> def g_with():
...     with ...:
...         pass
... 
>>> dis.dis(g_with)
  2           0 LOAD_CONST               1 (Ellipsis)
              3 SETUP_WITH               5 (to 11)
              6 POP_TOP

  3           7 POP_BLOCK
              8 LOAD_CONST               0 (None)
        >>   11 WITH_CLEANUP_START
             12 WITH_CLEANUP_FINISH
             13 END_FINALLY
             14 LOAD_CONST               0 (None)
             17 RETURN_VALUE

Here, co_cleanup would include a single 2-tuple with '0x03' as an absolute offset (for the setup) and '0x08' as a relative offset (for the cleanup). If '(f_lasti - 0x03) < 0x08' then __del__ on a generator-iterator or coroutine would raise ResourceWarning.

>>> def g_finally():
...     try:
...         pass
...     finally:
...         pass
... 
>>> dis.dis(g_finally)
  2           0 SETUP_FINALLY            4 (to 7)

  3           3 POP_BLOCK
              4 LOAD_CONST               0 (None)

  5     >>    7 END_FINALLY
              8 LOAD_CONST               0 (None)
             11 RETURN_VALUE

Here, co_cleanup would include a single 2-tuple with '0x00' as an absolute offset (for the setup) and '0x07' as a relative offset (for the cleanup). If '(f_lasti - 0x00) < 0x07' then __del__ on a generator-iterator or coroutine would raise ResourceWarning.

The key is that *only* __del__ would get slower (not explicit close() calls), and it would only get slower in cases where the frame was still suspended *and* co_cleanup was populated.
History
Date User Action Args
2016-11-14 11:37:44ncoghlansetrecipients: + ncoghlan, vstinner, njs, martin.panter, yselivanov
2016-11-14 11:37:44ncoghlansetmessageid: <1479123464.82.0.469895350315.issue28629@psf.upfronthosting.co.za>
2016-11-14 11:37:44ncoghlanlinkissue28629 messages
2016-11-14 11:37:44ncoghlancreate