Author ag6502
Date 2006-12-30.12:47:55
SpamBayes Score
Marked as misclassified
I added the two missing parts and the patch should be now functionally complete; now there co_names reordering pass during compilation so that all names used in LOAD_GLOBAL/LOAD_ATTR are moved at the beginning of the tuple, this allows the lookup cache to be allocated only for the needed size. The reordering is done *before* actually assembling basic blocks to avoid to deal with the change of opcode size after reordering.
Also now the timestamp is reset when rolling over to 0 and a sweep is done on all dictionaries and all code objects to clear cached lookups; I didn't implemented it in the gc because: 1) I'm not sure if all code objects and dictionaries are listed (how do I find code objects ?), 2) to do the traversing the code must be in gcmodule, or a richer interface must be exported, 3) what if garbage collecting is disabled ?

The extra cost for dictionary creation is normally (i.e. when a dictionary is recovered from the free pool) just setting the timestamp to 1, the extra cost for dictionary update is higher but around the "noise" of timeit.Timer on my system. The memory cost is three more 32-bit words for every dict and 4 + n*3 more words for every code object where n is the number of elements of co_names that are used in LOAD_GLOBAL/LOAD_ATTR opcodes. The patch as it stands don't need to change marshalling of compiled code (the loaded code objects will simply be "unoptimized" so n will be 1+max(oparg) for all arguments of LOAD_GLOBAL/LOAD_ATTR - wasting some cache slots).

The patch itself doesn't do anything unless the symbol CACHED_LOOKUPS is defined (this will cache LOAD_GLOBALs) or both CACHED_LOOKUPS and CACHED_MODULE_LOOKUPS are defined (this will cache both LOAD_GLOBALs and LOAD_ATTRs when the searched object is a module). This allows to have this optimization as an optional build option (does this make sense ?).
File Added: cached_lookups_12.patch
Date User Action Args
2007-08-23 15:55:46adminlinkissue1616125 messages
2007-08-23 15:55:46admincreate