Title: Argument Clinic: inline PyArg_UnpackTuple and PyArg_ParseStack(AndKeyword)?
Type: performance Stage:
Components: Argument Clinic Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: haypo, inada.naoki, larry, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-02-02 12:50 by haypo, last changed 2017-02-02 13:02 by serhiy.storchaka.

Messages (2)
msg286778 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-02 12:50
Argument Clinic calls one the following functions depending on parameters:

* PyArg_UnpackTuple(), _PyArg_UnpackStack()
* PyArg_ParseTuple(), _PyArg_ParseStack()
* PyArg_ParseTupleAndKeywords(), _PyArg_ParseStackAndKeywords()
* etc.

Would it make sense to emit C code instead of calling complex and slow PyArg_ParseXXX() functions? It would emit the most efficient C code to parse arguments.

I don't recall where this idea comes from. Maybe Larry Hastings told me once that he wants to implement this idea :-) I'm sure that Larry has a big plan but lacks time to implement all of his cool ideas.

Using profiled guided optimization (PGO), the compiler should be able to easily detect that error cases are unlikely and mark these code paths as unlikely.

We should probably experiment an implementation to be able to measure the speedup, to be able to say if the idea is worth it or not, in term of performance, since the motivation here is clearly performance.

We can begin with format strings only made of "O" format. Most simple example with divmod(), replace:

    if (!_PyArg_UnpackStack(args, nargs, "divmod", 2, 2, &x, &y)) { return NULL; }

with something like:

    if (nargs != 2) { _PyArg_ErrNumArgs(nargs, 2, 2, "divmod"); return NULL; }
    x = args[0];
    y = args[1];

The next question is if we should go further with more complex formats. Example with the format() function, replace:

    if (!_PyArg_ParseStack(args, nargs, "O|U:format", &value, &format_spec)) { ... }


    if (nargs < 1 || nargs > 2) { _PyArg_ErrNumArgs(nargs, 1, 2, "format"); return NULL; }

    value = args[0];

    if (nargs == 2) {
        format_spec = args[1];
        if (!PyUnicode_Check(format_spec)) { .. raise an exception ...; return NULL; }
        /* getargs.c calls PyUnicode_READY(), we should also do it here */
        if (PyUnicode_READY(format_spec) == -1) { return NULL; }
    else {
        format_spec = NULL;
msg286779 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-02 13:02
See issue23867 for the first step (unfinished). After that I planned to inline parsing code for PyArg_ParseTuple() with multiple arguments. Then split PyArg_ParseTupleAndKeywords() on two parts: first unpack keywords to linear sparse array and then handle it as positional arguments.
Date User Action Args
2017-02-02 13:02:24serhiy.storchakasetmessages: + msg286779
2017-02-02 12:50:49haypocreate