classification
Title: Argument Clinic: improve generated parser for 1-argument functions
Type: enhancement Stage: resolved
Components: Argument Clinic, Build Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: 23501 Superseder:
Assigned To: serhiy.storchaka Nosy List: josh.r, larry, pitrou, python-dev, scoder, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-02-20 14:40 by serhiy.storchaka, last changed 2015-04-04 14:07 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
clinic_meth_o.patch serhiy.storchaka, 2015-02-20 14:40 review
Messages (11)
msg236288 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-20 14:40
Proposed patch improve generated parsers for functions with single positional argument. Now they always generated as METH_O and PyArg_Parse() is used to parse single argument.

To avoid code churn in this and following changes it would be worth to extract all generated code in separated files.
msg236313 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-02-20 17:38
I'm not opposed to the patch in principle.  I assume your goal is to make Python faster--do you have any data on how much faster?

I don't support immediately changing all uses of Argument Clinic to generate their code into a separate file.  I would want to see a consensus from the community first.  Perhaps we should discuss it (again?) on python-dev?
msg236315 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-20 17:52
This is one step on long way. Second step will be to inline PyArg_Parse for 
some format codes ("i", "U", "y*", "O&", "O!"). Then we could try to expand 
PyArg_ParseTuple, at least for simple common cases. Then 
PyArg_ParseTupleAndKeywords. All this step will produce large diffs for 
generated code.
msg236367 - (view) Author: Stefan Behnel (scoder) * Date: 2015-02-21 15:36
Serhiy, I suggest you look at the code that Cython generates for its functions. It has been extensively profiled and optimised (years ago), so generating the same code for the argument clinic should yield the same performance.

And while I don't have exact numbers at hand, avoiding the tuple packing for the call by passing it into a METH_O function can make a substantial difference. It also kills support for keyword arguments, though.
msg236372 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-02-21 16:10
Stefan: Serhiy's patch only affects functions taking a single positional-only parameter.
msg236382 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-02-21 18:07
lgtm
msg236385 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-21 18:37
> Serhiy, I suggest you look at the code that Cython generates for its functions. It has been extensively profiled and optimised (years ago), so generating the same code for the argument clinic should yield the same performance.

Thanks, I'll look on it.

> And while I don't have exact numbers at hand, avoiding the tuple packing for the call by passing it into a METH_O function can make a substantial difference.

Good idea. Here are samples:

$ ./python -m timeit "chr(0x20ac)"
Unpatched: 1000000 loops, best of 3: 0.976 usec per loop
Patched:   1000000 loops, best of 3: 0.752 usec per loop

$ ./python -m timeit -s "from cmath import isnan; x = 1j" -- "isnan(x)"
Unpatched: 1000000 loops, best of 3: 0.62 usec per loop
Patched:   1000000 loops, best of 3: 0.386 usec per loop

Of course for more complex functions the effect is smaller.
msg236388 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-02-21 20:02
After expanding PyArg_Parse for "i" and "D" codes above tests give following results:

$ ./python -m timeit "chr(0x20ac)"
1000000 loops, best of 3: 0.558 usec per loop

$ ./python -m timeit -s "from cmath import isnan; x = 1j" -- "isnan(x)"
1000000 loops, best of 3: 0.278 usec per loop

About twice in comparison with initial variant!
msg236408 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-02-22 11:21
Let's make Argument Clinic a fierce optimizer!
(+1 on this)
msg240038 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-03 21:12
New changeset e10ad4d4d490 by Serhiy Storchaka in branch 'default':
Issue #23492: Argument Clinic now generates argument parsing code with
https://hg.python.org/cpython/rev/e10ad4d4d490
msg240072 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-04 14:07
New changeset 973c9ec53bbb by Serhiy Storchaka in branch 'default':
Fixed the array module broken in issue #23492.
https://hg.python.org/cpython/rev/973c9ec53bbb
History
Date User Action Args
2015-04-04 14:07:58python-devsetmessages: + msg240072
2015-04-03 22:00:38serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-04-03 21:12:35python-devsetnosy: + python-dev
messages: + msg240038
2015-02-25 15:30:28serhiy.storchakasetcomponents: + Argument Clinic
2015-02-22 11:21:40pitrousetnosy: + pitrou
messages: + msg236408
2015-02-22 11:06:40serhiy.storchakasetassignee: serhiy.storchaka
dependencies: + Argument Clinic: generate code into separate files by default
2015-02-21 20:02:04serhiy.storchakasetmessages: + msg236388
2015-02-21 18:37:21serhiy.storchakasetmessages: + msg236385
2015-02-21 18:07:15larrysetmessages: + msg236382
2015-02-21 16:10:03larrysetmessages: + msg236372
2015-02-21 15:36:58scodersetnosy: + scoder
messages: + msg236367
2015-02-21 00:49:41josh.rsetnosy: + josh.r
2015-02-20 17:52:01serhiy.storchakasetmessages: + msg236315
2015-02-20 17:38:35larrysetmessages: + msg236313
2015-02-20 14:40:11serhiy.storchakacreate