Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle unpacking of */** arguments and rvalues in the compiler #83501

Closed
markshannon opened this issue Jan 13, 2020 · 5 comments
Closed

Handle unpacking of */** arguments and rvalues in the compiler #83501

markshannon opened this issue Jan 13, 2020 · 5 comments
Labels
3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@markshannon
Copy link
Member

BPO 39320
Nosy @vstinner, @markshannon, @bharel, @pablogsal, @brandtbucher
PRs
  • bpo-39320: Handle unpacking of *values in compiler #17984
  • bpo-39320: Handle unpacking of **values in compiler #18141
  • bpo-39320: Don't use an oparg for new DICT_* opcodes. #18243
  • bpo-39320: Fix changes in the evaluation logic of unpacking operations. #18264
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-08-30.09:13:18.070>
created_at = <Date 2020-01-13.12:28:10.197>
labels = ['interpreter-core', '3.9', 'performance']
title = 'Handle unpacking of */** arguments and rvalues in the compiler'
updated_at = <Date 2021-12-18.15:41:31.066>
user = 'https://github.com/markshannon'

    bugs.python.org fields:

    activity = <Date 2021-12-18.15:41:31.066>
actor = 'bar.harel'
assignee = 'none'
closed = True
closed_date = <Date 2021-08-30.09:13:18.070>
closer = 'Mark.Shannon'
components = ['Interpreter Core']
creation = <Date 2020-01-13.12:28:10.197>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 39320
keywords = ['patch']
message_count = 5.0
messages = ['359901', '360548', '360749', '375260', '408857']
nosy_count = 5.0
nosy_names = ['vstinner', 'Mark.Shannon', 'bar.harel', 'pablogsal', 'brandtbucher']
pr_nums = ['17984', '18141', '18243', '18264']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue39320'
versions = ['Python 3.9']
    @markshannon
    Copy link
    Member Author

    Currently the unpacking of starred values in arguments and the right hand side of assignments is handled in the interpreter without any help from the compiler.
    The layout of arguments and values is visible to the compiler, so the compiler should do more of the work.

    We can replace the complex bytecodes used in unpacking with simpler more focused ones.
    Specifically the collection building operations
    BUILD_LIST_UNPACK, BUILD_TUPLE_UNPACK, BUILD_SET_UNPACK and BUILD_TUPLE_UNPACK_WITH_CALL
    can be replaced with simpler, and self-explanatory operations:
    LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE

    In addition, the mapping operations
    BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL
    can be replaced with DICT_UPDATE and DICT_MERGE.

    DICT_MERGE is like DICT_UPDATE but raises an exception for duplicate keys.

    This change would not have much of an effect of performance, as the bytecodes listed are relatively rarely used, but shrinking the interpreter is always beneficial.

    @markshannon markshannon added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Jan 13, 2020
    @markshannon
    Copy link
    Member Author

    New changeset 13bc139 by Mark Shannon in branch 'master':
    bpo-39320: Handle unpacking of *values in compiler (GH-17984)
    13bc139

    @markshannon
    Copy link
    Member Author

    New changeset 8a4cd70 by Mark Shannon in branch 'master':
    bpo-39320: Handle unpacking of **values in compiler (GH-18141)
    8a4cd70

    @vstinner
    Copy link
    Member

    These changes introduced a regression: bpo-41531 "Python 3.9 regression: Literal dict with > 65535 items are one item shorter".

    @bharel
    Copy link
    Mannequin

    bharel mannequin commented Dec 18, 2021

    Does this count as a regression or as an unintended bugfix for evaluation order?

    https://stackoverflow.com/a/70404659/1658617

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @vstinner @markshannon