there is a crash log inside the archive

The title is somewhat misleading, this is a C program that uses dlopen to load the python framework. 

I will look into this, but it is just as likely that code that loads or uses the python framework has a problem.

Hi Ronald,

Thank you for looking into it, and sorry for the misleading title.

You will find a crash log and some instructions on how to reproduce the problem inside the archive.

Thanks,

Marc

On 8 Jul 2010, at 10:18, Ronald Oussoren wrote:

Ronald Oussoren <ronaldoussoren@mac.com<mailto:ronaldoussoren@mac.com>> added the comment:

The title is somewhat misleading, this is a C program that uses dlopen to load the python framework.

I will look into this, but it is just as likely that code that loads or uses the python framework has a problem.

----------

_______________________________________
Python tracker <report@bugs.python.org<mailto:report@bugs.python.org>>
<http://bugs.python.org/issue9197>
_______________________________________

Marc Palluat de Besset
Mac Software Engineer

email : marc.palluatdebesset@sophos.com<mailto:marc.palluatdebesset@sophos.com>
tel : +44 1235 544056
loc : E2 floor 2

SOPHOS - simply secure

________________________________
Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 348 3873 20.

I reproduce the problem on Linux (./configure --enable-shared), after I modified the source code a bit to directly use Python.h and to link with libpython3.2.so (no call to dlopen). In gdb the stack trace has exactly the same symbols as the attached crash log, so the issue is not specific to Mac frameworks.

The crash is inside _PyImport_LoadDynamicModule("_pickle"), when _PyImport_FixupExtension() calls Py_DECREF(def->m_base.m_copy), this certainly frees objects allocated in a previous incarnation of the interpreter, and segfaults in type_dealloc(): _PyObject_GC_UNTRACK(type).

In import.c::_PyImport_FixupExtension(), the Py_DECREF is preceded by a comment: """ /* Somebody already imported the module, 
   likely under a different name.
   XXX this should really not happen. */
Py_DECREF(def->m_base.m_copy);
"""
I removed this statement, and the program now runs correctly. I don't know about memory leaks though...

Removing the call to Py_DECREF is fishy. 

The cleanup of the interpreter state in Py_Finalize doesn't clean up all state when m_copy refers to a version from a previous instance of the interpreter.

Maybe the tp_dealloc of a module object clean up m_copy? I don't know enough of this code and its intended behavior to know if that would be a valid change.

We definitely need unit tests about embedded python interpreter, I think there are none.

Unassigning, this is not mac-specific and I don't plan to work on this in the near future.

(changing subject to better tell what this issue is about)

Amaury,

Can you post your Linux code reproducing the issue?  I would like to take a look, but don't want to dig through an XCode project.

Here are the files I used to reproduce the crash.
See "build.sh" for the build command, there may be better ways...

Does the patch for issue11321 fix this issue?

Merging this into issue11321 since that relates to the same underlying problem and includes a simpler test case.