Message44265
This is a new implementation of the performance boost
in PyMarshal_ReadLastObjectFromFile where the pyc file
data is loaded into memory before parsing.
The original logic is as follows:
- Get the file size.
- If the file size is 16K or less, allocate a 16k char
buffer on the call stack and load the file data into
the buffer.
- Otherwise, if file size is 256k or less, malloc a
temp char buffer from the heap and load the file data
into the buffer.
- Otherwise, don't use a buffer.
Allocating 16k on the call stack can cause a stack
overflow when doing nested imports in a secondary
thread. The default thread stack size on some systems
is quite small (64k on HP-UX!). Also,
allocating/freeing memory on the heap for this purpose
may not be warranted.
This patch proposes a new implentation that allocates a
static 256k char buffer and uses a thread lock to keep
it thread-safe. The lock/unlock function logic was
taken from import.c.
Note: performance tests show an improvment on some
platforms when the static char buffer is defined at the
file level instead of inside the function. |
|
Date |
User |
Action |
Args |
2007-08-23 15:28:19 | admin | link | issue770280 messages |
2007-08-23 15:28:19 | admin | create | |
|