classification
Title: Optimize _PyFunction_FastCallDict() for **kwargs
Type: performance Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: haypo, inada.naoki, python-dev, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-01-19 11:33 by haypo, last changed 2017-02-24 18:30 by haypo. This issue is now closed.

Messages (8)
msg285773 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-19 11:33
Example:
---
def func(x, y):
    print(x, y)

def proxy2(func, **kw):
    func(**kw)

def proxy1(func, **kw):
    proxy2(func, **kw)
---

The "proxy2(func, **kw)" call in proxy1() is currently inefficient: _PyFunction_FastCallDict() converts the dictionary into a C array [key1, value1, key2, value2, ...] and then _PyEval_EvalCodeWithName() rebuilds the dictionary from the C array.

Since "func(*args, **kw)" proxies are common in Python, especially to call the parent constructor when overriding __init__, I think that it would be interesting to optimize this code path.

I first expected that it was a regression of FASTCALL, but Python < 3.6 doesn't optimize this code neither.
msg285777 - (view) Author: INADA Naoki (inada.naoki) * (Python committer) Date: 2017-01-19 11:44
Since mutating kw dict shouldn't affect caller's dict, caller and callee can't share the dict.

One possible optimization is using PyDict_Copy() to copy the dict.
It can reduce number of hash() calls.
msg285782 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-19 12:09
> It can reduce number of hash() calls.

Keys are strings (even interned strings), and hashes are cached.

In most cases kw is empty or very small. I doubt any optimization can have significant effect.
msg286642 - (view) Author: Roundup Robot (python-dev) Date: 2017-02-01 16:05
New changeset fcba9c9d987b by Victor Stinner in branch 'default':
Document that _PyFunction_FastCallDict() must copy kwargs
https://hg.python.org/cpython/rev/fcba9c9d987b
msg286643 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-01 16:08
INADA Naoki: "Since mutating kw dict shouldn't affect caller's dict, caller and callee can't share the dict."

Oh, I knew that, it's not the first time that I propose to implement this optimization and that I got this answer !? So I added a comment for myself, to remind me not to propose this optimization anymore :-D

I close this issue.


Serhiy Storchaka: "In most cases kw is empty or very small. I doubt any optimization can have significant effect."

I have no opinion on Naoki's optimization idea.

Naoki: Feel free to implement your idea to measure the speedup, but please open a new issue for that if you consider that it's worth it.

FYI Naoki and me started to list ideas to optimize CPython 3.7:
http://faster-cpython.readthedocs.io/cpython37.html
msg286654 - (view) Author: Roundup Robot (python-dev) Date:
New changeset 2c145e9eda756aa2001282d7c016e740dd00a2d7 by Victor Stinner in branch 'master':
Document that _PyFunction_FastCallDict() must copy kwargs
https://github.com/python/cpython/commit/2c145e9eda756aa2001282d7c016e740dd00a2d7
msg286661 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-01 17:20
> Changes by Roundup Robot <devnull@psf.upfronthosting.co.za>:

Why Roundup Robot spams with empty or almost empty messages?
msg286666 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-01 17:28
Serhiy Storchaka: "Why Roundup Robot spams with empty or almost empty messages?"

It smells like the migration to GitHub started :-)
History
Date User Action Args
2017-02-24 18:30:40hayposetresolution: fixed -> rejected
2017-02-01 17:28:37hayposetmessages: + msg286666
2017-02-01 17:20:02serhiy.storchakasetmessages: + msg286661
2017-02-01 17:00:21python-devsetstage: resolved
2017-02-01 17:00:21python-devsetresolution: rejected -> fixed
2017-02-01 17:00:21python-devsetmessages: + msg286654
2017-02-01 16:08:45hayposetstatus: open -> closed
resolution: rejected
2017-02-01 16:08:07hayposetmessages: + msg286643
2017-02-01 16:05:01python-devsetnosy: + python-dev
messages: + msg286642
2017-01-19 12:09:18serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg285782
2017-01-19 11:44:38inada.naokisetmessages: + msg285777
2017-01-19 11:33:01haypocreate