classification
Title: code objects should conserve memory
Type: resource usage Stage:
Components: Versions: Python 2.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, nnorwitz
Priority: normal Keywords: patch

Created on 2008-02-25 02:41 by nnorwitz, last changed 2008-03-15 22:47 by nnorwitz. This issue is now closed.

Files
File name Uploaded Description Edit
marshal-mem.patch nnorwitz, 2008-02-25 02:41 reduce code objects in memory from marshal.
Messages (3)
msg62965 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-02-25 02:41
Various bits are often duplicated in code objects.  For example,
sometimes names and varnames are equal.  In this case, we don't need two
objects since they are both const.  This patch implements a trivial fix
for this case.  However, there are more cases.  We should profile where
the memory is being used and do simple/cheap consolidations where
possible.  Another example would be a 1-element tuple containing:
(None,) for consts.

Some (all?) of these sorts of optimizations should probably be done in
the code object itself.
msg63085 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-02-27 22:08
Since names and varnames are known to be tuples of interned strings, 
they can be compared without calling abstract API. Just compare sizes 
and then compare item pointers in a loop.

What are the cases when co_names == co_varnames?  How often is this the 
case in the wild?
msg63568 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2008-03-15 22:47
Marshal is the wrong place for this sort of thing (the code object is
where it should be done).

I botched the analysis.  The case is common, but only for the empty
tuple which I forgot to ignore.  (None,) was a common case when I
measured it.  We should measure the actual memory savings before
anything is implemented.

To answer your question, the case were this happens is:

  def foo(self):  return self.self
History
Date User Action Args
2008-03-15 22:47:47nnorwitzsetstatus: open -> closed
resolution: rejected
messages: + msg63568
keywords: patch, patch
2008-02-27 22:08:08belopolskysetnosy: + belopolsky
messages: + msg63085
2008-02-25 02:41:40nnorwitzcreate