classification
Title: optimize out local variables at end of function
Type: resource usage Stage:
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: nnorwitz Nosy List: BreamoreBoy, arigo, nnorwitz, pitrou
Priority: normal Keywords: patch

Created on 2008-02-25 02:00 by nnorwitz, last changed 2010-07-20 13:43 by BreamoreBoy. This issue is now closed.

Files
File name Uploaded Description Edit
opt-out-local-var.patch nnorwitz, 2008-02-25 02:00 peepholer patch 1
Messages (8)
msg62957 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-02-25 02:00
This patch optimizes code like:

   x = any_expression
   return x

to:
   return any_expression

Currently it only optimizes out the local variable if there is a return
because it can't determine if this is the last use of the variable or not.

This shouldn't change behaviour under normal circumstances, but would
change behaviour with a debugger.  Perhaps this optimization should only
be performed if -O is passed on the command line?

This optimization saves two trips around the eval loop (STORE_FAST and
LOAD_FAST) and 6 bytes in the byte code.
msg62958 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-02-25 02:03
I forgot to mention that if another loop was added to PyCode_Optimize
that kept track of the # of times each local variable was
LOAD_FAST/STORE_FAST/DELETE_FAST and that the count was 2, we could
perform a similar optimization without requiring the return.

Bonus points for other cases like if it was the last use inside a list
comprehension or the variable is otherwise unaccessible.
msg62966 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-02-25 03:22
Guido says to do it only with -O. 
http://mail.python.org/pipermail/python-dev/2008-February/077193.html
msg63073 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2008-02-27 16:21
I suppose you are aware that performing this optimization in general
would break a lot of existing code that uses inspect.getstack() or
sys._getframe() to peek at the caller's local variables.  I know this
because it's one thing that Psyco doesn't do correctly, and one of the
most common causes I'm aware of for a random existing program to break
under Psyco.
msg63074 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-02-27 16:32
>  I suppose you are aware that performing this optimization in general
>  would break a lot of existing code that uses inspect.getstack() or
>  sys._getframe() to peek at the caller's local variables.  I know this

Yes, with this optimization the variable might never be set or when
the function exits, the value would be set to the previous value.

Note that the current optimization only works just before a return and
only for local variables.  It doesn't generally optimize out
variables, although that would be a good next step.

>  because it's one thing that Psyco doesn't do correctly, and one of the
>  most common causes I'm aware of for a random existing program to break
>  under Psyco.

How often does this cause problems?  Do you view this as psyco's
problem or broken user code?

I don't view this any different that a C compiler optimizing out
variables.  It can make debugging harder since the symbols no longer
exist.  In this case the variable name is not removed from the
co_varnames even if it is the only reference.  That would also be
nice, but left for another patch.  Since this will only be used with
-O and is currently limited, this seems reasonable to me.  But I would
like to know if others disagree.
msg63090 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2008-02-28 13:45
I view this as a problem with Psyco, not with the user code.
An even deeper reason for which the general optimization would break
code is because it changes the lifetime of objects.  For example, Psyco
contains specific, user-requested support to make sure the following
kind of code works:

def myfunc():
    f = open('somewhere', 'r')
    fd = f.fileno()
    return os.fstat(fd)

At the moment Python guarantees that the file object is not closed
before the function exits.  The above code cannot be rewritten like this:

def bogus():
    fd = open('somewhere', 'r').fileno()
    # the file is auto-closed here and fd becomes invalid
    return os.fstat(fd)

I know it's bad style to write code relying on this property, but still
I want to make sure you are aware that it *will* introduce obscure
breakage in existing code.
msg86921 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-05-01 22:03
Given that it is unlikely to give any speedup in real-world code, I
don't think we should add complexity to the compiler. Recommend closing.
msg110896 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-20 13:43
Closing as nobody has commented.
History
Date User Action Args
2010-07-20 13:43:53BreamoreBoysetstatus: open -> closed

nosy: + BreamoreBoy
messages: + msg110896

keywords: patch, patch
resolution: wont fix
2009-05-01 22:03:41pitrousetkeywords: patch, patch
nosy: + pitrou
messages: + msg86921

2008-03-20 04:25:22jafosetpriority: normal
assignee: nnorwitz
keywords: patch, patch
2008-03-20 04:21:38jafosetmessages: - msg62962
2008-02-28 13:45:50arigosetkeywords: patch, patch
messages: + msg63090
2008-02-27 16:32:11nnorwitzsetmessages: + msg63074
2008-02-27 16:21:24arigosetkeywords: patch, patch
nosy: + arigo
messages: + msg63073
2008-02-25 03:22:39nnorwitzsetkeywords: patch, patch
messages: + msg62966
2008-02-25 02:22:43nnorwitzsetkeywords: patch, patch
messages: + msg62962
versions: + Python 2.6
2008-02-25 02:03:09nnorwitzsetkeywords: patch, patch
messages: + msg62958
2008-02-25 02:00:13nnorwitzcreate