➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-09-04 23:27 by zephyrus00jp, last changed 2022-04-11 14:59 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| PyMem_Free.txt | zephyrus00jp, 2019-09-05 01:28 | |||
| Messages (8) | |||
|---|---|---|---|
| msg351154 - (view) | Author: Chiaki Ishikawa (zephyrus00jp) | Date: 2019-09-04 23:27 | |
Hi, I am new to Python bug tracker, so my setting of the fields may be inadequate. If so, apologies in advance. I think Use After Free bug is a potential security issue and so wanted to report ASAP. First my environment: I am using Debian GNU/Linux, and its kernel version is: uname -a output, SMP Debian 4.19.37-6 (2019-07-18) x86_64 GNU/Linux Python version is: # python3 --version Python 3.7.4 mozilla thunderbird mail client testing framework uses python as a test driver, and when I was checking the local build of thunderbird under a test suite invoked from |make mozmill| under valgrind (a memory usage checker), the first thing I noticed is the following message from valgrind. PyObject_Free seems to access a memory location (4 octets) in an already freed block. This happened many times during the test. I have not bothered to look into the source code of python, but a seasoned developer should be able to figure out where such reference is made. From valgrind log: ==30354== Invalid read of size 4 ==30354== at 0x5A29FE: PyObject_Free (in /usr/bin/python3.7) ==30354== by 0x5B7337: ??? (in /usr/bin/python3.7) ==30354== by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7) ==30354== by 0x58DE19: PyType_Ready (in /usr/bin/python3.7) ==30354== by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7) ==30354== by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7) ==30354== by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7) ==30354== by 0x5E17EC: ??? (in /usr/bin/python3.7) ==30354== by 0x653D88: ??? (in /usr/bin/python3.7) ==30354== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7) ==30354== by 0x4ACB09A: (below main) (libc-start.c:308) ==30354== Address 0x4c8b020 is 16 bytes after a block of size 576 free'd ==30354== at 0x4833FC0: free (vg_replace_malloc.c:538) ==30354== by 0x5B7337: ??? (in /usr/bin/python3.7) ==30354== by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7) ==30354== by 0x58DE19: PyType_Ready (in /usr/bin/python3.7) ==30354== by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7) ==30354== by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7) ==30354== by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7) ==30354== by 0x5E17EC: ??? (in /usr/bin/python3.7) ==30354== by 0x653D88: ??? (in /usr/bin/python3.7) ==30354== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7) ==30354== by 0x4ACB09A: (below main) (libc-start.c:308) ==30354== Block was alloc'd at ==30354== at 0x4832E13: malloc (vg_replace_malloc.c:307) ==30354== by 0x5A4B16: PyObject_Malloc (in /usr/bin/python3.7) ==30354== by 0x5B72BD: ??? (in /usr/bin/python3.7) ==30354== by 0x5BBBFF: PyDict_SetItem (in /usr/bin/python3.7) ==30354== by 0x58DE19: PyType_Ready (in /usr/bin/python3.7) ==30354== by 0x6482A0: _Py_ReadyTypes (in /usr/bin/python3.7) ==30354== by 0x63551A: _Py_InitializeCore_impl (in /usr/bin/python3.7) ==30354== by 0x6357AA: _Py_InitializeCore (in /usr/bin/python3.7) ==30354== by 0x5E17EC: ??? (in /usr/bin/python3.7) ==30354== by 0x653D88: ??? (in /usr/bin/python3.7) ==30354== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7) ==30354== by 0x4ACB09A: (below main) (libc-start.c:308) ==30354== TIA |
|||
| msg351155 - (view) | Author: Chiaki Ishikawa (zephyrus00jp) | Date: 2019-09-04 23:52 | |
Addtion:
PyObject_Realloc also has the issue of Use After Free: this may be more serious.
==31128== Invalid read of size 4
==31128== at 0x5A48CA: PyObject_Realloc (in /usr/bin/python3.7)
==31128== by 0x5DD8FB: _PyBytes_Resize (in /usr/bin/python3.7)
==31128== by 0x4F53BC: ??? (in /usr/bin/python3.7)
==31128== by 0x5D9A22: _PyMethodDef_RawFastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C75F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C61F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x54F3F2: PyEval_EvalCode (in /usr/bin/python3.7)
==31128== by 0x6313D1: ??? (in /usr/bin/python3.7)
==31128== by 0x631486: PyRun_FileExFlags (in /usr/bin/python3.7)
==31128== by 0x6320EE: PyRun_SimpleFileExFlags (in /usr/bin/python3.7)
==31128== by 0x653EED: ??? (in /usr/bin/python3.7)
==31128== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==31128== by 0x4ACB09A: (below main) (libc-start.c:308)
==31128== Address 0x5b21020 is 32,208 bytes inside a block of size 32,801 free'd
==31128== at 0x4835259: realloc (vg_replace_malloc.c:834)
==31128== by 0x5A49AB: PyObject_Realloc (in /usr/bin/python3.7)
==31128== by 0x5DD8FB: _PyBytes_Resize (in /usr/bin/python3.7)
==31128== by 0x4F53BC: ??? (in /usr/bin/python3.7)
==31128== by 0x5D9A22: _PyMethodDef_RawFastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C75F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C61F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x54F3F2: PyEval_EvalCode (in /usr/bin/python3.7)
==31128== by 0x6313D1: ??? (in /usr/bin/python3.7)
==31128== by 0x631486: PyRun_FileExFlags (in /usr/bin/python3.7)
==31128== by 0x6320EE: PyRun_SimpleFileExFlags (in /usr/bin/python3.7)
==31128== by 0x653EED: ??? (in /usr/bin/python3.7)
==31128== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==31128== by 0x4ACB09A: (below main) (libc-start.c:308)
==31128== Block was alloc'd at
==31128== at 0x4832E13: malloc (vg_replace_malloc.c:307)
==31128== by 0x5DE3AA: PyBytes_FromStringAndSize (in /usr/bin/python3.7)
==31128== by 0x4F5375: ??? (in /usr/bin/python3.7)
==31128== by 0x5D9A22: _PyMethodDef_RawFastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C75F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54FA9B: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x5DA6E1: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54C61F: ??? (in /usr/bin/python3.7)
==31128== by 0x5537DA: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x5DA3EB: _PyFunction_FastCallKeywords (in /usr/bin/python3.7)
==31128== by 0x54F8FB: _PyEval_EvalFrameDefault (in /usr/bin/python3.7)
==31128== by 0x54D0A1: _PyEval_EvalCodeWithName (in /usr/bin/python3.7)
==31128== by 0x54F3F2: PyEval_EvalCode (in /usr/bin/python3.7)
==31128== by 0x6313D1: ??? (in /usr/bin/python3.7)
==31128== by 0x631486: PyRun_FileExFlags (in /usr/bin/python3.7)
==31128== by 0x6320EE: PyRun_SimpleFileExFlags (in /usr/bin/python3.7)
==31128== by 0x653EED: ??? (in /usr/bin/python3.7)
==31128== by 0x65424D: _Py_UnixMain (in /usr/bin/python3.7)
==31128== by 0x4ACB09A: (below main) (libc-start.c:308)
==31128==
{
|
|||
| msg351163 - (view) | Author: Chiaki Ishikawa (zephyrus00jp) | Date: 2019-09-05 01:28 | |
PyMem_Free and PyMem_Realloc also suffer from similar "Use After Free" issue. The standard version of Python3.7 interpreter does not seem to have debug symbol and so I installed a debug version of it. Now it seems to have a shadow symbol of inlined function so that we can now learn the line numbers of functions where the problem occurs in more detail. I am attaching the valgrind log excerpts for PyObj_Free, PyObj_Realloc, PyMem_Free, PyMem_Realloc, etc. Sorry for piecemeal reporting. The earlier messages swamp the log very verbosely and so I had to disable the reporting one by one. PS: There seem to be cases where the result of reading uninitialized value is used for conditional testing, too. |
|||
| msg351165 - (view) | Author: Chiaki Ishikawa (zephyrus00jp) | Date: 2019-09-05 01:30 | |
I found that the inlined functions are used by other functions and resulted in similar issues. Once obmalloc.c is fixed, I think such problems are automatically fixed. |
|||
| msg351170 - (view) | Author: Tim Peters (tim.peters) *  |
Date: 2019-09-05 01:47 | |
You're probably chasing ghosts ;-) Please read about what needs to be done to use valgrind successfully with Python: https://github.com/python/cpython/blob/master/Misc/README.valgrind |
|||
| msg351253 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2019-09-06 13:55 | |
By default, Python is optimized for performance but its pymalloc memory allocator causes false alarms in Valgrind. > https://github.com/python/cpython/blob/master/Misc/README.valgrind This or you can disable pymalloc at runtime using: PYTHONMALLOC=malloc python3 ... or: PYTHONMALLOC=malloc_debug python3 ... See: https://docs.python.org/dev/c-api/memory.html and https://docs.python.org/dev/using/cmdline.html#envvar-PYTHONMALLOC |
|||
| msg351254 - (view) | Author: Charalampos Stratakis (cstratak) * | Date: 2019-09-06 14:27 | |
Another thing also, is to be sure to utilize the python's suppression file by adding the --suppressions=<filename> to valgrind's command line invocation. |
|||
| msg388303 - (view) | Author: STINNER Victor (vstinner) * |
Date: 2021-03-08 20:56 | |
It sounds like the reporter didn't use Valgrind properly. See last comments. If Valgrind still reports issues when it's used properly, please reopen the issue (or open a new issue). I close the issue for now. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:59:19 | admin | set | github: 82214 |
| 2021-03-08 20:56:52 | vstinner | set | status: open -> closed resolution: not a bug messages: + msg388303 stage: resolved |
| 2019-09-06 14:27:44 | cstratak | set | nosy:
+ cstratak messages: + msg351254 |
| 2019-09-06 13:55:06 | vstinner | set | nosy:
+ vstinner messages: + msg351253 |
| 2019-09-05 01:47:34 | tim.peters | set | nosy:
+ tim.peters messages: + msg351170 title: Use After Free: PyObject_Free -> Use After Free: PyObject_Free (valgrind) |
| 2019-09-05 01:30:00 | zephyrus00jp | set | messages: + msg351165 |
| 2019-09-05 01:28:47 | zephyrus00jp | set | files:
+ PyMem_Free.txt messages: + msg351163 |
| 2019-09-04 23:52:29 | zephyrus00jp | set | messages: + msg351155 |
| 2019-09-04 23:27:04 | zephyrus00jp | create | |