Memory leaks in Python on Windows #76207
Comments
|
I would like to report memory leaks in Python 3.6.3 when embedded in a C++ Windows app. I have download the Python 3.6.3 code and compiled a debug version of Python with my copy of Visual Studio 2017 Professional (have also confirmed the problem in Visual Studio 2015 Professional) and built the debug version of the Python DLL. With the following sample app I get memory leaks when compiled in debug mode: //Pull in the Windows header and the MSVC Debug C Runtime
#include <Windows.h>
#define _CRTDBG_MAP_ALLOC
#include <stdlib.h>
#include <crtdbg.h>
//Pull in Python
#include <Python.h>
int __stdcall WinMain(HINSTANCE /*hInstance*/, HINSTANCE /*hPrevInstance*/, _In_ LPTSTR /*lpCmdLine*/, int /*nCmdShow*/)
{
//Track memory leaks using the MSVC Debug CRT
_CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF);
//_CrtSetBreakAlloc(81);
//Initialize Python
Py_Initialize();
PyEval_InitThreads();//Pretend this app does something! //Shutdown Python
Py_FinalizeEx();
}Taking out the calls to Py_Initialize, PyEval_InitThreads & Py_FinalizeEx results in no memory leaks being reported. The actual reported memory leaks are reported to the output window in Visual Studio as follows: Detected memory leaks! If you uncomment the call to _CrtSetBreakAlloc and set the parameter to this API to for example 81, the Visual Studio debugger will break where the memory allocation occurs. I have taken all the numbers and confirmed these are memory allocations which occur in the Python functions I am using. Most of these are due to memory allocations of static C variables in the Python source code. I would be most grateful if these issues can be fixed up in the Python codebase. I am available to answer any questions about the issue at any time if necessary. Regards, |
|
Not sure how much we can do about this, but you may want to try with Python 3.7. Eric and Nick have been working on streamlining startup, which includes a lot of work (thanks Eric!) to remove static variables. Don't expect a fix in Python 3.6 for this, and don't be surprised if Eric/Nick closes this and references whichever bug they're using to track the other work, but it is something that we're aware of and are (slowly) working to address. |
|
OK, Thanks for the quick reply. I have just migrated my application from Python 3.3 to Python 3.6.3 and would not expect you to backport changes from the 3.7 development stream to the released stream. What I will do when I get a chance is pull down the 3.7 sources and compare the demo app I have produced with 3.7 vs 3.6.3 to see if improvements have been done in this area. |
|
I think it's reasonable to keep this open if the problems can be reproduced with the current 3.7 dev branch - while we've refactored a lot of things, we haven't specifically gone memory leak hunting for the embedded case, and I'm pretty sure there are still some items where we delay cleaning them up until the next Py_Initialize call. The relevant embedding test case is the one at https://github.com/python/cpython/blob/master/Programs/_testembed.c#L41, but I'm not sure we regularly run that in a way that would reliably detect memory leaks (it runs in a subprocess, rather than the main process, so the refleak hunting runs may not be handling it properly). |
|
I don't think the refleak handling would detect leaks across PyInitialize/PyFinalize calls anyway. Really, we probably just need to loop Initialize/Finalize a few hundred times and measure the memory usage of the process before/after. That will handle a range of leaks, though it doesn't necessarily help us track them down. |
|
Python doesn't clean everything at exit: see bpo-29881. I just modified Py_Main() in bpo-32030 to clean a few more bytes (like program_name): see pymain_free() in Modules/main.c. |
|
It's basically a duplicate of bpo-1635741. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: