Ok, I was guessing it was memory corruption, or something else nasty.

I've managed to eliminate subversion, so this just leaves apache, mod_python and python. I've rebuild with --with-pydebug and it's made the crash happen earlier. It appears to die now in some checking code that only happens when pydebug is used:

Program terminated with signal 6, Aborted.
#0  0xfefa0218 in _lwp_kill () from /usr/lib/libc.so.1
(gdb) bt
#0  0xfefa0218 in _lwp_kill () from /usr/lib/libc.so.1
#1  0xfef50c88 in raise () from /usr/lib/libc.so.1
#2  0xfef36e60 in abort () from /usr/lib/libc.so.1
#3  0xfe2af8d8 in Py_FatalError (msg=0xfe309618 "Invalid thread state for this thread") at Python/pythonrun.c:1559
#4  0xfe2a8990 in PyThreadState_Swap (newts=0x1e8b78) at Python/pystate.c:320
#5  0xfe2593b0 in PyEval_AcquireThread (tstate=0x1e8b78) at Python/ceval.c:252
#6  0xfe644b78 in get_interpreter () from /usr/local/libexec/apache2/mod_python.so
#7  0xfe648f28 in python_cleanup_handler () from /usr/local/libexec/apache2/mod_python.so
#8  0xff2a3e48 in run_cleanups () from /usr/local/lib/libapr-1.so.0
#9  0xff2a4674 in apr_pool_destroy () from /usr/local/lib/libapr-1.so.0
#10 0x0004581c in ap_process_http_connection ()
#11 0x000415e4 in ap_run_process_connection ()
#12 0x0004c7f8 in child_main ()
#13 0x0004cb18 in make_child ()
#14 0x0004cc14 in startup_children ()
#15 0x0004d778 in ap_mpm_run ()
#16 0x00027110 in main ()
(gdb) f 5
#5  0xfe2593b0 in PyEval_AcquireThread (tstate=0x1e8b78) at Python/ceval.c:252
252             if (PyThreadState_Swap(tstate) != NULL)
(gdb) l
247             if (tstate == NULL)
248                     Py_FatalError("PyEval_AcquireThread: NULL new thread state");
249             /* Check someone has called PyEval_InitThreads() to create the lock */
250             assert(interpreter_lock);
251             PyThread_acquire_lock(interpreter_lock, 1);
252             if (PyThreadState_Swap(tstate) != NULL)
253                     Py_FatalError(
254                             "PyEval_AcquireThread: non-NULL old thread state");
255     }
256     
(gdb) f 4
#4  0xfe2a8990 in PyThreadState_Swap (newts=0x1e8b78) at Python/pystate.c:320
320                             Py_FatalError("Invalid thread state for this thread");
(gdb) l
315                        to it, we need to ensure errno doesn't change.
316                     */
317                     int err = errno;
318                     PyThreadState *check = PyGILState_GetThisThreadState();
319                     if (check && check->interp == newts->interp && check != newts)
320                             Py_FatalError("Invalid thread state for this thread");
321                     errno = err;
322             }
323     #endif
324             return oldts;
(gdb) f 3
#3  0xfe2af8d8 in Py_FatalError (msg=0xfe309618 "Invalid thread state for this thread") at Python/pythonrun.c:1559
1559            abort();
(gdb) l
1554            OutputDebugString("\n");
1555    #ifdef _DEBUG
1556            DebugBreak();
1557    #endif
1558    #endif /* MS_WINDOWS */
1559            abort();
1560    }
1561    
1562    /* Clean up and exit */
1563    
(gdb) 

So your guess about threads might well be right. I am compiling with --with-threads. I guess the next step would be to disable threads?

Tim.