This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python cApi memory problem. Py_Initialize memory leak
Type: resource usage Stage:
Components: Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: rstarostecki, skrah, vstinner
Priority: normal Keywords:

Created on 2013-12-05 08:50 by rstarostecki, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pytest.tgz rstarostecki, 2013-12-05 08:50 test code , valgrind output
vgrind3.3.3vc.out rstarostecki, 2013-12-05 12:48
vgrindNext.out rstarostecki, 2013-12-05 14:14
Messages (10)
msg205283 - (view) Author: Roman (rstarostecki) Date: 2013-12-05 08:50
I wrote small test program using Python cApi, and found some memory issues while working on it. I checked it with valgrind. (test code and valgrind output appended)
Maybe I'm doing something wrong, but most of the problem occurs durring Py_Initialize. I've checked python3.2.5 and 3.3.3 version. The latest version is much less confusing, but I can still see there something nasty. 
I would be grateful if you could look at this, and tell me If I'm doing something wrong, or if I can do something to prevent this memory issues.
msg205288 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-05 11:56
Did you use --suppressions=Misc/valgrind-python.supp?
msg205289 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-05 12:00
Also, you have zero "definitely lost". "possibly lost" is not
particularly informative in the context of running Python:
These are almost certainly false positives.
msg205290 - (view) Author: Roman (rstarostecki) Date: 2013-12-05 12:32
I didn't use --suppressions=Misc/valgrind-python.supp before . (But I've done it now). Nothing important has changed.
I understand that "possibly lost is not particularly informative". I'm  rather worried about "Invalid read of size 4" etc. Isn't it potentially dangerous? I thought it is some kind of memory corruption, isn't it?
msg205291 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-05 12:35
Did you compile Python --with-valgrind?
msg205292 - (view) Author: Roman (rstarostecki) Date: 2013-12-05 12:48
I've just done it. Python 3.3.3 --with-valgrind. I can't see the difference. Output appended.
msg205299 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-05 13:58
You shoud configure --with-valgrind *and* use the suppression list, or
disable pymalloc using configure.

Victor
msg205301 - (view) Author: Roman (rstarostecki) Date: 2013-12-05 14:14
I compiled python --with-valgrind --without-pymalloc, and used valgrind with suppressions. 

valgrind --suppressions=../Misc/valgrind-python.supp --leak-check=full --show-reachable=no --show-possibly-lost=no --track-origins=yes --log-file=vgrindNext.out ./test
msg205624 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-12-09 00:30
Sorry, but I still don't understand this issue.

"Invalid read of size 4" is a known false positive. It can be worked around using ./configure --with-valgrind and the suppression list, or using ./configure --without-pymalloc. If you still get the warning, you used the wrong options or you are still using another Python binary (or shared library). Make sure that you are linked to your newly compiled shared library.

"valgrind --leak-check=full --log-file=valgrind.log ./python -c pass" shows me "possibly lost: 286,779 bytes in 654 blocks". This is also another known issue: Python doesn't release all the memory at exit, they are many "singletons" and variables initialized once but never released.

The PEP 3121 helps this issue but the PEP is not fully implemented yet, many modules should still be modified.

So what is your question? Do you think that Python leaks memory? Why do you think so?
msg205678 - (view) Author: Roman (rstarostecki) Date: 2013-12-09 11:32
I've checked it one more time.
And you're right (Sorry for trouble). I left old pyconfig.h in one place, so my new python compilation was not just what I wanted. 

Now I belive that everything with memory is ok. 

Thank you very much for your help.
History
Date User Action Args
2022-04-11 14:57:55adminsetgithub: 64092
2013-12-09 11:46:34vstinnersetresolution: not a bug
2013-12-09 11:32:59rstarosteckisetstatus: open -> closed

messages: + msg205678
2013-12-09 00:30:50vstinnersetmessages: + msg205624
2013-12-05 14:14:27rstarosteckisetfiles: + vgrindNext.out

messages: + msg205301
2013-12-05 13:58:02vstinnersetmessages: + msg205299
2013-12-05 12:48:49rstarosteckisetfiles: + vgrind3.3.3vc.out

messages: + msg205292
2013-12-05 12:35:58skrahsetmessages: + msg205291
2013-12-05 12:32:54rstarosteckisetmessages: + msg205290
2013-12-05 12:00:28skrahsetmessages: + msg205289
2013-12-05 11:56:57skrahsetmessages: + msg205288
2013-12-05 11:41:56pitrousetnosy: + vstinner, skrah
2013-12-05 08:50:23rstarosteckicreate