This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: apparent memory leak using ctypes
Type: resource usage Stage:
Components: ctypes Versions: Python 3.5, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, blindgoat, martin.panter
Priority: normal Keywords:

Created on 2015-10-29 19:16 by blindgoat, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
ctype_stressor.py blindgoat, 2015-10-29 19:16 python program showing the leak
Messages (4)
msg253688 - (view) Author: Martin Smith (blindgoat) * Date: 2015-10-29 19:16
Attached file demonstrates a memory leak arising in ctypes (I think).  Leak occurs in both 2.7 and 3.5, though at different rates.

Humble apologies if I've screwed up somewhere.  Very nervous about submitting this.
msg253709 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-10-30 02:06
I am running your script with the Python 3.5.0 from Arch Linux on an x86-64 computer. The reported numbers slowly increase and then plateau at 221 million:

$ python ctype_stressor.py
       1  0.0000000e+00  3.8010880e+07
  100000  2.8538793e+00  1.4648934e+08
  200000  2.9961207e+00  1.5189606e+08
  300000  3.1383621e+00  1.5730278e+08
  400000  3.2806034e+00  1.6270950e+08
  500000  3.4512931e+00  1.6919757e+08
  600000  3.5935345e+00  1.7460429e+08
  700000  3.7357759e+00  1.8001101e+08
  800000  3.8780172e+00  1.8541773e+08
  900000  4.0202586e+00  1.9082445e+08
 1000000  4.1909483e+00  1.9731251e+08
 1100000  4.3331897e+00  2.0271923e+08
 1200000  4.4754310e+00  2.0812595e+08
 1300000  4.6176724e+00  2.1353267e+08
 1400000  4.7599138e+00  2.1893939e+08
 1500000  4.7883621e+00  2.2002074e+08
 1600000  4.7883621e+00  2.2002074e+08
. . .
 2700000  4.7883621e+00  2.2002074e+08
 2800000  4.8043103e+00  2.2062694e+08
. . .
 4100000  4.8043103e+00  2.2062694e+08
 4200000  4.8133621e+00  2.2097101e+08
. . .
13000000  4.8133621e+00  2.2097101e+08

Monitoring the process with “htop”, I see the total virtual memory (VIRT) ramping up to about 79 MiB, and then jumping back to about 62 MiB, and the resident physical memory (RES) ramps within 36–53 MiB in a similar pattern.

Similarly, Python 2.7.10 ramps within RES = 32–43 MiB, and your script is showing 177 million:

11400000  5.5962126e+00  1.7691443e+08

Is this the same as what you are seeing? How long do you have to you run your script for?

The memory figures in your script seem to be out by a factor of four, assuming the third column is meant to be a number of bytes. But I don’t think there is a memory leak. At worst there is an unexpected dependency cycle between objects, meaning the memory builds up a bit until the garbage collector runs.
msg253721 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2015-10-30 07:03
cast() always creates a ref cycle, which is probably what you are seeing.

>>> from ctypes import c_double, cast, POINTER
>>> samples = (c_double*23)()
>>> cast(samples, POINTER(c_double))
<__main__.LP_c_double object at 0x7f33bc920560>
>>> samples._objects
{139860183544448: <__main__.c_double_Array_23 object at 0x7f33bc8ef280>}
>>> samples
<__main__.c_double_Array_23 object at 0x7f33bc8ef280>
msg253741 - (view) Author: Martin Smith (blindgoat) * Date: 2015-10-30 15:56
I'm persuaded that this is not a sustained memory leak.

Thanks to Martin Panter and Benjamin Peterson for the tests and the careful explanations.

Apologies for distracting busy people.
History
Date User Action Args
2022-04-11 14:58:23adminsetgithub: 69698
2015-10-30 15:56:18blindgoatsetstatus: open -> closed
resolution: not a bug
messages: + msg253741
2015-10-30 07:03:26benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg253721
2015-10-30 02:06:50martin.pantersetnosy: + martin.panter

messages: + msg253709
versions: + Python 2.7
2015-10-29 19:16:36blindgoatcreate