Issue19972
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2013-12-13 10:14 by skrah, last changed 2022-04-11 14:57 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
issue19972.patch | skrah, 2013-12-13 10:40 |
Messages (10) | |||
---|---|---|---|
msg206044 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-13 10:14 | |
Hi Alexandre, the following leaks appear after 64c6d52793be. I'm not sure yet if they're caused or just exposed by the changeset. Code: ============== import sys import pickle sys.exit(0) ============== ==8118== 864 (192 direct, 672 indirect) bytes in 3 blocks are definitely lost in loss record 2,198 of 2,365 ==8118== at 0x4C28B8C: malloc (vg_replace_malloc.c:270) ==8118== by 0x41D12A: _PyMem_RawMalloc (obmalloc.c:60) ==8118== by 0x41D5E9: PyObject_Malloc (obmalloc.c:351) ==8118== by 0x533FC9: _PyObject_GC_Malloc (gcmodule.c:1726) ==8118== by 0x534092: _PyObject_GC_New (gcmodule.c:1749) ==8118== by 0x450CDF: new_dict (dictobject.c:391) ==8118== by 0x4527BB: _PyDict_NewPresized (dictobject.c:1043) ==8118== by 0x4DCAFD: PyEval_EvalFrameEx (ceval.c:2366) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4E5BF7: fast_function (ceval.c:4332) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x42619E: _PyObject_CallMethodIdObjArgs (abstract.c:2359) ==8118== by 0x502FBE: PyImport_ImportModuleLevelObject (import.c:1473) ==8118== by 0x4CC3EC: builtin___import__ (bltinmodule.c:210) ==8118== by 0x5B11D5: PyCFunction_Call (methodobject.c:99) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E4E01: PyEval_CallObjectWithKeywords (ceval.c:4101) ==8118== by 0x4DD718: PyEval_EvalFrameEx (ceval.c:2466) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4E5BF7: fast_function (ceval.c:4332) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== 2,112 (128 direct, 1,984 indirect) bytes in 2 blocks are definitely lost in loss record 2,301 of 2,365 ==8118== at 0x4C28B8C: malloc (vg_replace_malloc.c:270) ==8118== by 0x41D12A: _PyMem_RawMalloc (obmalloc.c:60) ==8118== by 0x41D5E9: PyObject_Malloc (obmalloc.c:351) ==8118== by 0x533FC9: _PyObject_GC_Malloc (gcmodule.c:1726) ==8118== by 0x4734D9: PyType_GenericAlloc (typeobject.c:784) ==8118== by 0x456438: dict_new (dictobject.c:2604) ==8118== by 0x473355: type_call (typeobject.c:751) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E6313: do_call (ceval.c:4454) ==8118== by 0x4E57B5: call_function (ceval.c:4252) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4E5BF7: fast_function (ceval.c:4332) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x42619E: _PyObject_CallMethodIdObjArgs (abstract.c:2359) ==8118== by 0x502FBE: PyImport_ImportModuleLevelObject (import.c:1473) ==8118== by 0x4CC3EC: builtin___import__ (bltinmodule.c:210) ==8118== by 0x5B11D5: PyCFunction_Call (methodobject.c:99) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E4E01: PyEval_CallObjectWithKeywords (ceval.c:4101) ==8118== by 0x4DD718: PyEval_EvalFrameEx (ceval.c:2466) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== ==8118== 2,402 (64 direct, 2,338 indirect) bytes in 1 blocks are definitely lost in loss record 2,304 of 2,365 ==8118== at 0x4C28B8C: malloc (vg_replace_malloc.c:270) ==8118== by 0x41D12A: _PyMem_RawMalloc (obmalloc.c:60) ==8118== by 0x41D5E9: PyObject_Malloc (obmalloc.c:351) ==8118== by 0x533FC9: _PyObject_GC_Malloc (gcmodule.c:1726) ==8118== by 0x534092: _PyObject_GC_New (gcmodule.c:1749) ==8118== by 0x450CDF: new_dict (dictobject.c:391) ==8118== by 0x450E4C: PyDict_New (dictobject.c:429) ==8118== by 0x45F97D: module_init (moduleobject.c:372) ==8118== by 0x47342B: type_call (typeobject.c:766) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E6313: do_call (ceval.c:4454) ==8118== by 0x4E57B5: call_function (ceval.c:4252) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x42619E: _PyObject_CallMethodIdObjArgs (abstract.c:2359) ==8118== by 0x502FBE: PyImport_ImportModuleLevelObject (import.c:1473) ==8118== by 0x4CC3EC: builtin___import__ (bltinmodule.c:210) ==8118== by 0x5B11D5: PyCFunction_Call (methodobject.c:99) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E4E01: PyEval_CallObjectWithKeywords (ceval.c:4101) ==8118== by 0x4DD718: PyEval_EvalFrameEx (ceval.c:2466) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4E5BF7: fast_function (ceval.c:4332) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E5ACA: fast_function (ceval.c:4322) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== ==8118== 4,919 (64 direct, 4,855 indirect) bytes in 1 blocks are definitely lost in loss record 2,347 of 2,365 ==8118== at 0x4C28B8C: malloc (vg_replace_malloc.c:270) ==8118== by 0x41D12A: _PyMem_RawMalloc (obmalloc.c:60) ==8118== by 0x41D5E9: PyObject_Malloc (obmalloc.c:351) ==8118== by 0x533FC9: _PyObject_GC_Malloc (gcmodule.c:1726) ==8118== by 0x534092: _PyObject_GC_New (gcmodule.c:1749) ==8118== by 0x450CDF: new_dict (dictobject.c:391) ==8118== by 0x450E15: new_dict_with_shared_keys (dictobject.c:420) ==8118== by 0x458477: _PyObjectDict_SetItem (dictobject.c:3746) ==8118== by 0x46215A: _PyObject_GenericSetAttrWithDict (object.c:1143) ==8118== by 0x462394: PyObject_GenericSetAttr (object.c:1185) ==8118== by 0x4618E4: PyObject_SetAttr (object.c:914) ==8118== by 0x4DABEB: PyEval_EvalFrameEx (ceval.c:2111) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x5A1348: method_call (classobject.c:347) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x482B2A: slot_tp_init (typeobject.c:5895) ==8118== by 0x47342B: type_call (typeobject.c:766) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E6313: do_call (ceval.c:4454) ==8118== by 0x4E57B5: call_function (ceval.c:4252) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4E5BF7: fast_function (ceval.c:4332) ==8118== by 0x4E5796: call_function (ceval.c:4250) ==8118== by 0x4DF8C7: PyEval_EvalFrameEx (ceval.c:2826) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x5A1348: method_call (classobject.c:347) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x42632D: PyObject_CallFunctionObjArgs (abstract.c:2381) ==8118== by 0x4DF288: PyEval_EvalFrameEx (ceval.c:2711) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x5B05F2: function_call (funcobject.c:632) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x42619E: _PyObject_CallMethodIdObjArgs (abstract.c:2359) ==8118== by 0x502FBE: PyImport_ImportModuleLevelObject (import.c:1473) ==8118== by 0x4CC3EC: builtin___import__ (bltinmodule.c:210) ==8118== by 0x5B11D5: PyCFunction_Call (methodobject.c:99) ==8118== by 0x4252E7: PyObject_Call (abstract.c:2087) ==8118== by 0x4E4E01: PyEval_CallObjectWithKeywords (ceval.c:4101) ==8118== by 0x4DD718: PyEval_EvalFrameEx (ceval.c:2466) ==8118== by 0x4E3340: PyEval_EvalCodeEx (ceval.c:3576) ==8118== by 0x4D3A59: PyEval_EvalCode (ceval.c:770) ==8118== by 0x4CD6BE: builtin_exec (bltinmodule.c:876) ==8118== by 0x5B11B3: PyCFunction_Call (methodobject.c:93) ==8118== by 0x4E6946: ext_do_call (ceval.c:4546) ==8118== by 0x4DFC60: PyEval_EvalFrameEx (ceval.c:2866) ==8118== |
|||
msg206051 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-13 10:40 | |
This patch fixes the leak, but needs review (I do not know the _pickle module very well). |
|||
msg206053 - (view) | Author: STINNER Victor (vstinner) * | Date: 2013-12-13 10:58 | |
According to the example in the PEP 3121, issue19972.patch is not needed. In practice, I don't see in PyImport_Cleanup() where tp_clear is called. This function calls: _PyModule_Clear(mod); Py_DECREF(mod); |
|||
msg206057 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-13 11:04 | |
See #11826 for more details where the freefunc is called. Perhaps the. docs or the PEP need an update. |
|||
msg206059 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-13 11:18 | |
The docs say: inquiry m_clear A clear function to call during GC clearing of the module object, or NULL if not needed. freefunc m_free A function to call during deallocation of the module object, or NULL if not needed. So I assume that GC clearing is not happening if sys.exit() is called. Note that the leak only occurs with sys.exit(). |
|||
msg206067 - (view) | Author: Antoine Pitrou (pitrou) * | Date: 2013-12-13 11:59 | |
Yeah, it sounds logical for m_free to do the clearing too (same as tp_dealloc vs. tp_clear). Note that the PEP 3121 scheme turns out to be problematic for proper resource management: https://mail.python.org/pipermail/python-dev/2013-August/127862.html |
|||
msg206146 - (view) | Author: Alexandre Vassalotti (alexandre.vassalotti) * | Date: 2013-12-13 21:01 | |
The patch is good. I am not sure if you need the freefunc cast though. The example in the PEP 3121 should updated if freefunc is actually required. I didn't define freefunc because of this example. |
|||
msg206153 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-13 22:21 | |
I just see that it should be: static void pickle_free(PyObject *m) ... Even then, the cast is necessary, otherwise you get this warning: /home/stefan/hg/cpython/Modules/_pickle.c:7450:1: warning: initialization from incompatible pointer type [enabled by default] /home/stefan/hg/cpython/Modules/_pickle.c:7450:1: warning: (near initialization for '_picklemodule.m_free') [enabled by default] I agree that it would be nice to update the PEP, but I guess that would be Martin's call. |
|||
msg206179 - (view) | Author: Roundup Robot (python-dev) | Date: 2013-12-14 12:43 | |
New changeset 8a78988fdb04 by Stefan Krah in branch 'default': Issue #19972: Add rarely used freefunc. This fixes a leak if sys.exit() http://hg.python.org/cpython/rev/8a78988fdb04 |
|||
msg206182 - (view) | Author: Stefan Krah (skrah) * | Date: 2013-12-14 13:26 | |
Should be fixed. Thanks for the comments everyone. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:57:55 | admin | set | github: 64171 |
2013-12-14 13:26:21 | skrah | set | status: open -> closed resolution: fixed messages: + msg206182 stage: patch review -> resolved |
2013-12-14 12:43:59 | python-dev | set | nosy:
+ python-dev messages: + msg206179 |
2013-12-13 22:21:49 | skrah | set | nosy:
+ loewis messages: + msg206153 |
2013-12-13 21:01:49 | alexandre.vassalotti | set | messages: + msg206146 |
2013-12-13 11:59:27 | pitrou | set | nosy:
+ ncoghlan, pitrou messages: + msg206067 |
2013-12-13 11:18:17 | skrah | set | messages: + msg206059 |
2013-12-13 11:04:45 | skrah | set | messages: + msg206057 |
2013-12-13 10:58:17 | vstinner | set | nosy:
+ vstinner messages: + msg206053 |
2013-12-13 10:45:33 | skrah | set | stage: patch review |
2013-12-13 10:40:43 | skrah | set | files:
+ issue19972.patch keywords: + patch messages: + msg206051 |
2013-12-13 10:14:34 | skrah | create |