This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Memory problems detected using Valgrind
Type: resource usage Stage: resolved
Components: Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, sxt1001
Priority: normal Keywords:

Created on 2022-02-24 14:11 by sxt1001, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
stack-information_python3.11.0a5.txt sxt1001, 2022-02-26 12:38
Messages (7)
msg413912 - (view) Author: tongxiaoge (sxt1001) Date: 2022-02-24 14:11
Reproduction steps:
1. Execute command: iotop -b -n 10 &
2. Execute the command in another session: valgrind /usr/sbin/iotop -b -n 5 > iotop_test

The output information is as follows:
[root@openEuler ~]# valgrind /usr/sbin/iotop -b -n 5 > iotop_test
==13750== Memcheck, a memory error detector
==13750== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==13750== Using Valgrind-3.16.0 and LibVEX; rerun with -h for copyright info
==13750== Command: /usr/sbin/iotop -b -n 5
==13750==
==13750== Conditional jump or move depends on uninitialised value(s)
==13750== at 0x49B2C40: PyUnicode_Decode (unicodeobject.c:3488)
==13750== by 0x49B335B: unicode_new (unicodeobject.c:15465)
==13750== by 0x4982C07: type_call (typeobject.c:1014)
==13750== by 0x492CA17: _PyObject_MakeTpCall (call.c:191)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:116)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:103)
==13750== by 0x48F1863: PyObject_Vectorcall (abstract.h:127)
==13750== by 0x48F1863: call_function (ceval.c:5075)
==13750== by 0x48F1863: _PyEval_EvalFrameDefault (ceval.c:3518)
==13750== by 0x48EAEE7: _PyEval_EvalFrame (pycore_ceval.h:40)
==13750== by 0x48EAEE7: function_code_fastcall (call.c:330)
==13750== by 0x492CBE7: _PyObject_FastCallDictTstate (call.c:118)
==13750== by 0x492CEEB: _PyObject_Call_Prepend (call.c:489)
==13750== by 0x498A007: slot_tp_init (typeobject.c:6964)
==13750== by 0x4982C4F: type_call (typeobject.c:1026)
==13750== by 0x492CA17: _PyObject_MakeTpCall (call.c:191)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:116)
==13750== by 0x48F1863: _PyObject_VectorcallTstate (abstract.h:103)
==13750== by 0x48F1863: PyObject_Vectorcall (abstract.h:127)
==13750== by 0x48F1863: call_function (ceval.c:5075)
==13750== by 0x48F1863: _PyEval_EvalFrameDefault (ceval.c:3518)
==13750==
==13751== Warning: invalid file descriptor 1024 in syscall close()
==13751== Warning: invalid file descriptor 1025 in syscall close()
==13751== Warning: invalid file descriptor 1026 in syscall close()
==13751== Warning: invalid file descriptor 1027 in syscall close()
==13751== Use --log-fd= to select an alternative log fd.
==13751== Warning: invalid file descriptor 1028 in syscall close()
==13751== Warning: invalid file descriptor 1029 in syscall close()
==13752== Warning: invalid file descriptor 1024 in syscall close()
==13752== Warning: invalid file descriptor 1025 in syscall close()
==13752== Warning: invalid file descriptor 1026 in syscall close()
==13752== Warning: invalid file descriptor 1027 in syscall close()
==13752== Use --log-fd= to select an alternative log fd.
==13752== Warning: invalid file descriptor 1028 in syscall close()
==13752== Warning: invalid file descriptor 1029 in syscall close()
==13750==
==13750== HEAP SUMMARY:
==13750== in use at exit: 1,069,715 bytes in 10,017 blocks
==13750== total heap usage: 589,638 allocs, 579,621 frees, 128,672,782 bytes allocated
==13750==
==13750== LEAK SUMMARY:
==13750== definitely lost: 0 bytes in 0 blocks
==13750== indirectly lost: 0 bytes in 0 blocks
==13750== possibly lost: 1,042,483 bytes in 9,894 blocks
==13750== still reachable: 27,232 bytes in 123 blocks
==13750== suppressed: 0 bytes in 0 blocks
==13750== Rerun with --leak-check=full to see details of leaked memory
==13750==
==13750== Use --track-origins=yes to see where uninitialised values come from
==13750== For lists of detected and suppressed errors, rerun with: -s
==13750== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Software version: python3-3.9.9 & python3.9.10  iotop-6.0

The above stack information is in Python 3.9.10 and this problem cannot be reproduced in Python 3.7.


So is it a python3  problem or an iotop problem?  How to fix it.
msg414042 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2022-02-25 21:21
It is possible that some core developer may get regular valgrind reports, but I would not know who.  I am pretty sure that you should run it at least on the latest release of the 'main' branch, 3.11.0a5, and perhaps even better, the current tip.  There might be a performance patch that was not backported.  (I have no idea what iotop is.)
msg414099 - (view) Author: tongxiaoge (sxt1001) Date: 2022-02-26 12:38
The latest version is also problematic (Python 3.11.0a5). The above is the output stack information. Here are the steps to reproduce:

1. iotop -b -n 10 &

2. valgrind --leak-check=full /usr/sbin/iotop -b -n 5 > iotop_ test
msg414100 - (view) Author: tongxiaoge (sxt1001) Date: 2022-02-26 12:41
I also installed Python3.7.9、Python3. 8.5 tested and found no problems.
msg414113 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2022-02-26 19:28
Posting a thousand line message makes following a discussion extremely difficult.  Move it into an attachment, if you did not do that already, and delete the message by 'editing' it.  As I said, I cannot otherwise respond to this.
msg414147 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2022-02-27 10:50
Valgrind does not understand Python's custom obmalloc allocator. You have to run valgrind with our suppression rules Misc/valgrind-python.supp to ignore false errors.
msg414197 - (view) Author: tongxiaoge (sxt1001) Date: 2022-02-28 14:14
Ok, thanks. There is no memory problem exposed when I add --suppressions=Misc/valgrind-python.supp. I will close this issue.
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 91005
2022-02-28 14:14:07sxt1001setstatus: open -> closed
resolution: not a bug
messages: + msg414197

stage: resolved
2022-02-27 15:11:54terry.reedysetnosy: - terry.reedy
2022-02-27 10:50:30christian.heimessetnosy: + christian.heimes
messages: + msg414147
2022-02-27 10:43:38christian.heimessetmessages: - msg414097
2022-02-26 19:28:21terry.reedysetmessages: + msg414113
2022-02-26 12:41:52sxt1001setmessages: + msg414100
2022-02-26 12:38:20sxt1001setfiles: + stack-information_python3.11.0a5.txt

messages: + msg414099
2022-02-26 12:33:33sxt1001setmessages: + msg414097
2022-02-25 21:21:23terry.reedysetnosy: + terry.reedy
messages: + msg414042
2022-02-24 14:11:26sxt1001create