This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python function call optimization: avoid temporary tuple to pass **kwargs
Type: performance Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: jdemeyer, methane, serhiy.storchaka, vstinner, xtreak
Priority: normal Keywords:

Created on 2018-07-10 23:13 by vstinner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg321400 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-10 23:13
On the following code, f() uses CALL_FUNCTION_EX bytecode to call g(). The bytecode loads 'kw' variable which is a dictionary. But internally, the dictionary is converted to a temporary tuple, and later a new dictionary is created. Maybe the temporary tuple could be avoided?

def g(*args, **kw):
    ...

def f(*args, **kw):
    g(*args, **kw)

In Python 3.6, before FASTCALL, CALL_FUNCTION_EX calls:

* do_call_core(): kw dict
* PyObject_Call(): kw dict
* function_call(): kw dict -> create a temporary tuple of keys and names: (key[0], value[0], ...)
* _PyEval_EvalCodeWithName(): if CO_VARKEYWORDS, rebuild a new dictionary for keyword arguments (**kw)

In Python master branch (future 3.8) with FASTCALL, CALL_FUNCTION_EX calls:

* do_call_core(): kw dict
* _PyFunction_FastCallDict(): kw dict -> a temporary tuple for keyword names ('kwnames') is created
* _PyEval_EvalCodeWithName(): if CO_VARKEYWORDS, rebuild a new dictionary for keyword arguments (**kw)

To be clear: FASTCALL didn't make this specific function call (Python => Python with **kw) worse nor better.
msg321558 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-12 14:41
Related issues with similar discussion in the past : 

https://bugs.python.org/issue27840
https://bugs.python.org/issue29318 - Similar discussion where PyDict_Copy was proposed

Thanks
msg321568 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-12 16:04
Yeah, I recall these issues since I wrote them :-D

The fix for bpo-29318 was to document that we must copy the kwargs dict ;-)

But this issue is not about the copy of the kwargs dict, but about two useless conversions: kwargs dict -> kwnames tuple -> kwargs dict.
msg339492 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-04-05 10:27
This might be solvable using PEP 580 by using METH_VARARGS instead of METH_FASTCALL for such functions. This would still require a temporary tuple for the positional args but no additional dict would need to be allocated.
msg402398 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-21 22:23
No activity for 2 years, I close the issue.
History
Date User Action Args
2022-04-11 14:59:03adminsetgithub: 78271
2021-09-21 22:23:04vstinnersetstatus: open -> closed
resolution: rejected
messages: + msg402398

stage: resolved
2019-04-05 10:27:28jdemeyersetnosy: + jdemeyer
messages: + msg339492
2018-07-12 16:04:36vstinnersetmessages: + msg321568
2018-07-12 14:41:59xtreaksetnosy: + xtreak
messages: + msg321558
2018-07-10 23:13:39vstinnercreate