This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Micro optimizations
Type: Stage:
Components: Interpreter Core Versions: Python 2.3
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: gvanrossum, nnorwitz, rhettinger
Priority: normal Keywords: patch

Created on 2002-05-27 21:33 by nnorwitz, last changed 2022-04-10 16:05 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
opt.diff nnorwitz, 2002-05-27 21:33 micro optimizations patch 1
opt.diff nnorwitz, 2002-09-13 15:19 patch 2 - 13 sep
opt3.diff nnorwitz, 2002-12-17 02:09 patch #3 updated to remove leaked refs
Messages (11)
msg40139 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-05-27 21:33
This is stuff I've had sitting around for a while.
I was attempting to improve performance in
some paths.

 * Most of the changes are from a loop -> memset.
 * intobject changes are to initialize small ints at
startup,
   so smallints don't have to be checked for each new int
 * other misc very small clean-ups

Please review and test to see if there are any
problems.  Also feedback whether this improves
performance for various platforms (tested on Linux)
or if this patch is even worth it.

Files modified are:  Include/intobject.h
Python/{ceval,pythonrun}.c
Objects/{tuple,list,int,frame,}object.c

All changes are independant, except for the int changes
which affect:  Include/intobject.h, Python/pythonrun.c,
and Objects/intobject.c.
It may also be useful to define the small negative int
(NSMALLNEGINTS) to be 5 or so instead of 1.  There are
several uses -2, -3, ... in the standard library.
msg40140 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2002-05-31 22:11
Logged In: YES 
user_id=80475

Wow, you plowed through a lot of code!

Two sets of optimizations look worthwhile, the memsets() 
and the XINCREFs to INCREFS.

Probably the fastlocals substitutions should be done also, 
but more for beauty and clarity than speed.

I checked those three categories of changes on my 
machine.  They compile fine, pass the standard regression 
tests and checkout okay on my personal, realcode 
testfarm.

I don't think the PyInt_Init() addition buys us anything.  
The register and macro tweaks may cost more in review 
time and potential errors than they could ever save in 
cumulative computer time.

Recommend you get these in before someone changes the 
codebase.

msg40141 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-06-05 21:43
Logged In: YES 
user_id=6380

I like all of these, even PyInt_Init(). Go for it.
msg40142 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-07-03 15:59
Logged In: YES 
user_id=33168

Checked in the memset()s in:
  {list,tuple}object.c and _sre.c.
object.c 2.178

Still have to do int and frame.
I've cleaned up int so that if there is an init
failure, a fatal error is raised similar to other
initializations.  I will get around to checking that in.
msg40143 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2002-09-08 04:57
Logged In: YES 
user_id=80475

Are these all done now?
msg40144 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-09-09 00:28
Logged In: YES 
user_id=33168

No, I've still got some outstanding modifications.
msg40145 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-09-13 15:19
Logged In: YES 
user_id=33168

I've updated the patch.  It's more of the same types of changes:
 * intobject stuff is the same, ie allocate small ints on
startup
   - the only change is to bump NSMALLNEGINTS from 1 to 5
 * frameobject: intern "__builtins__" on startup not in
Frame_New
 * DECREF instead of XDECREF in frame for code, builtins,
and globals
   (note, INCREF was used for these, not XINCREF)

I've been running with these for months, so I'm pretty sure
they are safe.
Unless you changed your mind about the int changes, these
changes should
be ok too.  Just giving you one last chance to change your
mind. :-)
msg40146 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-09-19 00:36
Logged In: YES 
user_id=6380

Setting the Resolution to Remind because it's not done IMO.

It's possible for an app embedding Python to call
Py_Initialize() more than once, with Py_Finalize() in
between. Then Py_Frame_Init() would leak a string and
PyInt_Init() would leak 105 int objects, plus the free list.
And, how do you know that all small ints fit in one freelist
block?

But please go on, I (still :-) like the idea.
msg40147 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-12-17 02:09
Logged In: YES 
user_id=33168

I updated the patch so that it no longer leaks references. 
The patch is missing the *_Init() calls in pythonrun.c  My
python run has a bunch of other patches, so I didn't update
it here.  While I believe the leaks from this patch are
fixed, there are still leaks in the other areas.  See
http://python.org/sf/613222.

We can decrease NSMALLPOSINTS from 100 to a smaller number.
 I don't know if anyone cares, though.
msg40148 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2002-12-30 16:50
Logged In: YES 
user_id=6380

I like this patch. Neal, can you check it in? Leave
NSMALLPOSINTS to 100. (How did you find that it can be
decreased?)
msg40149 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2002-12-30 22:31
Logged In: YES 
user_id=33168

Checked in as:
    Include/intobject.h : 2.27
    Include/pythonrun.h : 2.58
    Objects/intobject.c : 2.97
    Objects/frameobject.c : 2.71
    Python/pythonrun.c : 2.173
History
Date User Action Args
2022-04-10 16:05:21adminsetgithub: 36652
2002-05-27 21:33:41nnorwitzcreate