classification
Title: a faster Modulefinder
Type: Stage:
Components: Interpreter Core Versions: Python 2.5
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: theller
Priority: normal Keywords: patch

Created on 2005-11-11 11:51 by theller, last changed 2006-10-17 19:56 by theller. This issue is now closed.

Files
File name Uploaded Description Edit
modulefinder.patch theller, 2005-11-11 11:51 Speedup patch
Messages (3)
msg49008 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2005-11-11 11:51
py2exe uses Python's modulefinder to find the modules
and packages that belong to one or more scripts.

For not too small projects, the runtime of modulefinder
is quite long.  On my system, the time to find all 533
modules my project needs is around 48 seconds.

So, I profiled the Python 2.4 modulefinder, and patched
it for a speedup of a factor of ~2.5 - the time
required to find the modules drops to around 19 seconds.
msg49009 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2005-11-11 12:02
Logged In: YES 
user_id=11105

Here is a description of the changes in the patch:

Modulefinder's scan_code method did call ord() on each
character of the co.co_code string, that took the most time,
and it built the argument (again with ord() calls) of each
bytecode that had one, even if it was never used.

The patch changes the code to
- work on the characters of the co.co_code string, avoiding
the calls to ord() altogether
- create the bytecodes argument only when needed,
- create the bytecode with struct.pack which is faster.

I did not stop there, so other changes were that the objects
that scan_code needs most are passed as default arguments to
the functions instead of looking them up in the global
namespace.

This patch will probably be in the next py2exe release, so
it will undergo some testing.

I would appreciate comments on the patch.
msg49010 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2006-10-17 19:56
Logged In: YES 
user_id=11105

This patch is out of date.
History
Date User Action Args
2005-11-11 11:51:32thellercreate