This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Argument Clinic: inline PyArg_UnpackTuple and PyArg_ParseStack(AndKeyword)?
Type: performance Stage: resolved
Components: Argument Clinic Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: larry, methane, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-02-02 12:50 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (2)
msg286778 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-02-02 12:50
Argument Clinic calls one the following functions depending on parameters:

* PyArg_UnpackTuple(), _PyArg_UnpackStack()
* PyArg_ParseTuple(), _PyArg_ParseStack()
* PyArg_ParseTupleAndKeywords(), _PyArg_ParseStackAndKeywords()
* etc.

Would it make sense to emit C code instead of calling complex and slow PyArg_ParseXXX() functions? It would emit the most efficient C code to parse arguments.

I don't recall where this idea comes from. Maybe Larry Hastings told me once that he wants to implement this idea :-) I'm sure that Larry has a big plan but lacks time to implement all of his cool ideas.

Using profiled guided optimization (PGO), the compiler should be able to easily detect that error cases are unlikely and mark these code paths as unlikely.

We should probably experiment an implementation to be able to measure the speedup, to be able to say if the idea is worth it or not, in term of performance, since the motivation here is clearly performance.



We can begin with format strings only made of "O" format. Most simple example with divmod(), replace:

    if (!_PyArg_UnpackStack(args, nargs, "divmod", 2, 2, &x, &y)) { return NULL; }

with something like:

    if (nargs != 2) { _PyArg_ErrNumArgs(nargs, 2, 2, "divmod"); return NULL; }
    x = args[0];
    y = args[1];


The next question is if we should go further with more complex formats. Example with the format() function, replace:

    if (!_PyArg_ParseStack(args, nargs, "O|U:format", &value, &format_spec)) { ... }

with:

    if (nargs < 1 || nargs > 2) { _PyArg_ErrNumArgs(nargs, 1, 2, "format"); return NULL; }

    value = args[0];

    if (nargs == 2) {
        format_spec = args[1];
        if (!PyUnicode_Check(format_spec)) { .. raise an exception ...; return NULL; }
        /* getargs.c calls PyUnicode_READY(), we should also do it here */
        if (PyUnicode_READY(format_spec) == -1) { return NULL; }
    }
    else {
        format_spec = NULL;
    }
msg286779 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-02 13:02
See issue23867 for the first step (unfinished). After that I planned to inline parsing code for PyArg_ParseTuple() with multiple arguments. Then split PyArg_ParseTupleAndKeywords() on two parts: first unpack keywords to linear sparse array and then handle it as positional arguments.
History
Date User Action Args
2022-04-11 14:58:42adminsetgithub: 73605
2018-09-19 23:20:17vstinnersetstatus: open -> closed
resolution: out of date
stage: resolved
2017-02-02 13:02:24serhiy.storchakasetmessages: + msg286779
2017-02-02 12:50:49vstinnercreate