Message103370
It's definitely a stack overflow.
Most of the backtraces show an important number of frames.
The last frame is this:
#0 PyMarshal_ReadLastObjectFromFile (fp=0x13e8200)
at ../Python/marshal.c:1026
filesize = <value optimized out>
and a disassembly show us that the segfault is generated on a callq:
0x4bd4d6 <PyMarshal_ReadLastObjectFromFile+54>: callq 0x4168e8 <fileno@plt>
And if you look at the code, it's obvious what's happening:
PyObject *
PyMarshal_ReadLastObjectFromFile(FILE *fp)
{
/* 75% of 2.1's .pyc files can exploit SMALL_FILE_LIMIT.
* REASONABLE_FILE_LIMIT is by defn something big enough for Tkinter.pyc.
*/
#define SMALL_FILE_LIMIT (1L << 14)
#define REASONABLE_FILE_LIMIT (1L << 18)
#ifdef HAVE_FSTAT
off_t filesize;
#endif
#ifdef HAVE_FSTAT
filesize = getfilesize(fp);
if (filesize > 0) {
char buf[SMALL_FILE_LIMIT];
char* pBuf = NULL;
if (filesize <= SMALL_FILE_LIMIT)
pBuf = buf;
else if (filesize <= REASONABLE_FILE_LIMIT)
pBuf = (char *)PyMem_MALLOC(filesize);
if (pBuf != NULL) {
[...]
}
SMALL_FILE_LIMIT is 1 << 14 which is roughly 16K (not that reasonable :-).
So when we enter PyMarshal_ReadLastObjectFromFile and allocate buf, we push around 16K on the stack, which is a lot. That's why we segfault soon after when we call a function (callq), there's no space left on the stack.
So there are several solutions:
- make buf static, but it would increase Python size by almost 16K
- reduce SMALL_FILE_LIMIT, or remove it altogether. I guess SMALL_FILE_LIMIT is there to speedup loading of small files, but I'm not sure that malloc() would incur an important overhead
- reading the whole file to memory sounds weird, we sould probably be using mmap() here
Peers ? |
|
Date |
User |
Action |
Args |
2010-04-16 21:59:02 | neologix | set | recipients:
+ neologix, liang, Thomas.Smith |
2010-04-16 21:59:02 | neologix | set | messageid: <1271455142.75.0.373735125193.issue7332@psf.upfronthosting.co.za> |
2010-04-16 21:59:01 | neologix | link | issue7332 messages |
2010-04-16 21:58:59 | neologix | create | |
|