This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author christian.heimes
Recipients christian.heimes, eric.snow, gvanrossum, kumaraditya, vstinner
Date 2022-01-27.14:47:05
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1643294825.42.0.75112046433.issue46449@roundup.psfhosted.org>
In-reply-to
Content
The problem is in PyImport_ImportFrozenModuleObject -> unmarshal_frozen_code() -> frozen_info.get_code() -> _Py_get_importlib__bootstrap_external_toplevel() call chain.

PyImport_ImportFrozenModuleObject() expects unmarshal_frozen_code() to return a strong reference to the code object. However a frozen_info struct with a get_code() function returns a borrowed reference from deepfreeze.c's toplevel code object.

# --- test.c
#include <Python.h>
int main(int argc, char *argv[])
{
    for (int i=1; i <= 100; i++) {
        Py_SetProgramName(L"./_testembed");
        Py_Initialize();
        Py_Finalize();
        printf("Loop #%d: %zd refs, bootstrap refs: %zd\n", i, _Py_RefTotal, Py_REFCNT(_Py_get_importlib__bootstrap_external_toplevel()));
    }
}
# ---

$ gcc -IInclude -I. -o test test.c libpython3.11d.a -lm && ./test
Loop #1: -3 refs, bootstrap refs: 999999998
Loop #2: -8 refs, bootstrap refs: 999999997
Loop #3: -13 refs, bootstrap refs: 999999996
Loop #4: -18 refs, bootstrap refs: 999999995
Loop #5: -23 refs, bootstrap refs: 999999994
Loop #6: -28 refs, bootstrap refs: 999999993
Loop #7: -33 refs, bootstrap refs: 999999992
Loop #8: -38 refs, bootstrap refs: 999999991
Loop #9: -43 refs, bootstrap refs: 999999990
Loop #10: -48 refs, bootstrap refs: 999999989


After I changed unmarshal_frozen_code() to "return Py_NewRef(code);", the reference count of the frozen bootstrap module stays stable, but total refcount increases over time:

Loop #1: 10 refs, bootstrap refs: 999999999
Loop #2: 18 refs, bootstrap refs: 999999999
Loop #3: 26 refs, bootstrap refs: 999999999
Loop #4: 34 refs, bootstrap refs: 999999999
Loop #5: 42 refs, bootstrap refs: 999999999
Loop #6: 50 refs, bootstrap refs: 999999999
Loop #7: 58 refs, bootstrap refs: 999999999
Loop #8: 66 refs, bootstrap refs: 999999999
Loop #9: 74 refs, bootstrap refs: 999999999
History
Date User Action Args
2022-01-27 14:47:05christian.heimessetrecipients: + christian.heimes, gvanrossum, vstinner, eric.snow, kumaraditya
2022-01-27 14:47:05christian.heimessetmessageid: <1643294825.42.0.75112046433.issue46449@roundup.psfhosted.org>
2022-01-27 14:47:05christian.heimeslinkissue46449 messages
2022-01-27 14:47:05christian.heimescreate