classification
Title: Possible resource warning in "with open()"
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Mark.Shannon, benjamin.peterson, jwilk, miss-islington, ncoghlan, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2018-07-07 17:11 by serhiy.storchaka, last changed 2018-07-13 16:19 by jwilk. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8159 merged serhiy.storchaka, 2018-07-07 17:21
PR 8197 merged miss-islington, 2018-07-09 12:41
PR 8198 merged serhiy.storchaka, 2018-07-09 12:51
Messages (5)
msg321224 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-07 17:11
The bytecode generated for "with open()":

with open(path) as file:
    data = file.read()

  1           0 LOAD_NAME                0 (open)
              2 LOAD_NAME                1 (path)
              4 CALL_FUNCTION            1
              6 SETUP_WITH              14 (to 22)
              8 STORE_NAME               2 (file)

  2          10 LOAD_NAME                2 (file)
             12 LOAD_METHOD              3 (read)
             14 CALL_METHOD              0
             16 STORE_NAME               4 (data)
             18 POP_BLOCK
             20 BEGIN_FINALLY
        >>   22 WITH_CLEANUP_START
             24 WITH_CLEANUP_FINISH
             26 END_FINALLY
             28 LOAD_CONST               0 (None)
             30 RETURN_VALUE

The execution can be interrupted by Ctrl-C between calling open() and entering the 'with' block. In this case the file object will be created, but its __enter__ and __exit__ methods will be not executed. As a result it will be closed after disappearing a reference to it and a ResourceWarning will be emitted.

The solution is disabling interruption before the SETUP_WITH opcode. It is already disabled before SETUP_FINALLY and YIELD_FROM. It is worth to disable it before BEFORE_ASYNC_WITH for consistency although I don't have examples for it.

See also issue29988.
msg321288 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-08 20:07
Maybe it is worth to disable interrupting before more opcodes. For example for fixing a problem with the contextlib.nullcontext example (issue34067) we need to disable interrupting before STORE_FAST, LOAD_FAST and JUMP_FORWARD.
msg321314 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-07-09 12:40
New changeset 3f4d90d4d72921f16babd3f52d7df804916af224 by Nick Coghlan (Serhiy Storchaka) in branch 'master':
bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
https://github.com/python/cpython/commit/3f4d90d4d72921f16babd3f52d7df804916af224
msg321316 - (view) Author: miss-islington (miss-islington) Date: 2018-07-09 13:31
New changeset f5197ddfd0f2c5da848af57196448810bd18bb82 by Miss Islington (bot) in branch '3.7':
bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
https://github.com/python/cpython/commit/f5197ddfd0f2c5da848af57196448810bd18bb82
msg321328 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-09 16:02
New changeset eeaae26ce5abce56292330c29459337d6bd57aaf by Serhiy Storchaka in branch '3.6':
[3.6] bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159) (GH-8198)
https://github.com/python/cpython/commit/eeaae26ce5abce56292330c29459337d6bd57aaf
History
Date User Action Args
2018-07-13 16:19:58jwilksetnosy: + jwilk
2018-07-09 16:03:41serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-07-09 16:02:28serhiy.storchakasetmessages: + msg321328
2018-07-09 13:31:07miss-islingtonsetnosy: + miss-islington
messages: + msg321316
2018-07-09 12:51:37serhiy.storchakasetpull_requests: + pull_request7750
2018-07-09 12:41:28miss-islingtonsetpull_requests: + pull_request7749
2018-07-09 12:40:17ncoghlansetmessages: + msg321314
2018-07-08 20:07:04serhiy.storchakasetmessages: + msg321288
2018-07-07 17:21:22serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request7723
2018-07-07 17:11:52serhiy.storchakacreate