Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible resource warning in "with open()" #78247

Closed
serhiy-storchaka opened this issue Jul 7, 2018 · 5 comments
Closed

Possible resource warning in "with open()" #78247

serhiy-storchaka opened this issue Jul 7, 2018 · 5 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@serhiy-storchaka
Copy link
Member

BPO 34066
Nosy @ncoghlan, @benjaminp, @jwilk, @markshannon, @serhiy-storchaka, @1st1, @miss-islington
PRs
  • bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. #8159
  • [3.7] bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159) #8197
  • [3.6] bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159) #8198
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-07-09.16:03:41.491>
created_at = <Date 2018-07-07.17:11:52.350>
labels = ['interpreter-core', '3.7', '3.8', 'performance']
title = 'Possible resource warning in "with open()"'
updated_at = <Date 2018-07-13.16:19:58.631>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2018-07-13.16:19:58.631>
actor = 'jwilk'
assignee = 'none'
closed = True
closed_date = <Date 2018-07-09.16:03:41.491>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2018-07-07.17:11:52.350>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 34066
keywords = ['patch']
message_count = 5.0
messages = ['321224', '321288', '321314', '321316', '321328']
nosy_count = 7.0
nosy_names = ['ncoghlan', 'benjamin.peterson', 'jwilk', 'Mark.Shannon', 'serhiy.storchaka', 'yselivanov', 'miss-islington']
pr_nums = ['8159', '8197', '8198']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue34066'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']
    @serhiy-storchaka
    Copy link
    Member Author

    The bytecode generated for "with open()":

    with open(path) as file:
    data = file.read()

    1 0 LOAD_NAME 0 (open)
    2 LOAD_NAME 1 (path)
    4 CALL_FUNCTION 1
    6 SETUP_WITH 14 (to 22)
    8 STORE_NAME 2 (file)

    2 10 LOAD_NAME 2 (file)
    12 LOAD_METHOD 3 (read)
    14 CALL_METHOD 0
    16 STORE_NAME 4 (data)
    18 POP_BLOCK
    20 BEGIN_FINALLY
    >> 22 WITH_CLEANUP_START
    24 WITH_CLEANUP_FINISH
    26 END_FINALLY
    28 LOAD_CONST 0 (None)
    30 RETURN_VALUE

    The execution can be interrupted by Ctrl-C between calling open() and entering the 'with' block. In this case the file object will be created, but its __enter__ and __exit__ methods will be not executed. As a result it will be closed after disappearing a reference to it and a ResourceWarning will be emitted.

    The solution is disabling interruption before the SETUP_WITH opcode. It is already disabled before SETUP_FINALLY and YIELD_FROM. It is worth to disable it before BEFORE_ASYNC_WITH for consistency although I don't have examples for it.

    See also bpo-29988.

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Jul 7, 2018
    @serhiy-storchaka
    Copy link
    Member Author

    Maybe it is worth to disable interrupting before more opcodes. For example for fixing a problem with the contextlib.nullcontext example (bpo-34067) we need to disable interrupting before STORE_FAST, LOAD_FAST and JUMP_FORWARD.

    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Jul 9, 2018

    New changeset 3f4d90d by Nick Coghlan (Serhiy Storchaka) in branch 'master':
    bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
    3f4d90d

    @miss-islington
    Copy link
    Contributor

    New changeset f5197dd by Miss Islington (bot) in branch '3.7':
    bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
    f5197dd

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset eeaae26 by Serhiy Storchaka in branch '3.6':
    [3.6] bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159) (GH-8198)
    eeaae26

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @ncoghlan @serhiy-storchaka @miss-islington