classification
Title: Speed up function calls/can add more introspection info
Type: performance Stage:
Components: Interpreter Core Versions: Python 3.2
process
Status: closed Resolution: later
Dependencies: Superseder:
Assigned To: Nosy List: collinwinter, jyasskin, nnorwitz, terry.reedy
Priority: normal Keywords: patch

Created on 2005-01-23 18:32 by nnorwitz, last changed 2010-06-17 01:46 by collinwinter. This issue is now closed.

Files
File name Uploaded Description Edit
py-method.patch nnorwitz, 2005-01-23 18:32 patch 1
py-method2m.patch nnorwitz, 2005-01-24 23:28 patch 2 - minimal
py-method2all.patch nnorwitz, 2005-01-24 23:38 patch 2 - all (much larger than 2m)
Messages (5)
msg47599 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-01-23 18:32
This patch adds a new method type (flags) METH_ARGS
(yeah, the name could be better) that is used in
PyMethodDef.  METH_ARGS means the min and max # of
arguments are specified in the PyMethodDef by adding 2
new fields.  This information can be used in ceval to
call the method.  No tuple packing/unpacking is
required since the C stack is used.

The original patch only modifies Python/bltinmodule.c.
 If the approach is desirable, Objects/*.c should be
modified and so should code in Modules/ (probably).

The benefits are:
 * faster function calls
 * simplify function call machinery by removing
METH_NOARGS, METH_O, and possibly METH_VARARGS
 * more introspection info for C functions (ie, min/max
arg count)

The primary drawback is:
 * the defn of the MethodDef (# args) is separate from
the function defn
 * potentially more error prone to write C methods???

I've measured between 13-22% speed improvement when
doing simple tests like:

  ./python ./Lib/timeit.py -v 'pow(3, 5)'

I think the difference tends to be fairly constant at
about .3 usec per loop.

I'm not sure of the effect on memory usage.  I wouldn't
expect it to be much in either direction.

Note:  This patch does not make the min/max arg count
available to Python code.  If this patch is accepted,
that seems like it should also be done.

It's possible that METH_VARARGS may not be able to go away.
msg47600 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-01-23 18:40
Logged In: YES 
user_id=33168

Also see,
http://mail.python.org/pipermail/python-dev/2005-January/051251.html
msg47601 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-01-24 23:23
Logged In: YES 
user_id=33168

Martin pointed out that chr(5.3) is mishandled by this patch
and needs to be corrected.
msg107980 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-06-17 01:26
This appears to be a feature request that could now go into 3.2 at the earliest, or even 3.3 if it were to violate the core change moratorium. But I do not know if it even applies to the 3.x series.
msg107982 - (view) Author: Collin Winter (collinwinter) * (Python committer) Date: 2010-06-17 01:46
I tried making this work early last year as part of Unladen Swallow, and even though I got it working and it does speed up certain builtin calls, it didn't move overall application performance at all. I believe this was due to cache effects, branch mispredicts or something else. Based on that experience, I don't believe this is really worth pursuing on its own.

That said, something like this is included in Unladen Swallow, which uses it to make direct calls to certain C functions. I'm going to close this accordingly.
History
Date User Action Args
2010-06-17 01:46:33collinwintersetstatus: open -> closed
type: enhancement -> performance
resolution: later
messages: + msg107982
2010-06-17 01:26:16terry.reedysetversions: + Python 3.2, - Python 2.5
nosy: + terry.reedy

messages: + msg107980

type: enhancement
2008-10-11 19:31:46collinwintersetnosy: + collinwinter
2008-03-03 06:20:13jyasskinsetnosy: + jyasskin
2005-01-23 18:32:06nnorwitzcreate