classification
Title: Memory leak with CTypes Structure
Type: resource usage Stage: resolved
Components: ctypes Versions:
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: a01, alex, lunaryorn, skrah
Priority: normal Keywords:

Created on 2011-09-17 06:29 by a01, last changed 2011-11-03 18:09 by skrah. This issue is now closed.

Messages (6)
msg144170 - (view) Author: M (a01) Date: 2011-09-17 06:29
When using Python 2.5.4 on Windows (and potentially other versions and platforms), usage of CTypes like the following cause a memory leak.  The memory leak can be verified with ProcessExplorer, as the memory Python takes up keeps increasing until eventually Python will get a MemoryError.
----
import ctypes
def hi():
    class c1(ctypes.Structure):
            _fields_=[('f1',ctypes.c_uint8)]
    class c2(ctypes.Structure):
            _fields_=[('g1',c1*2)]

while True:
    test=hi()
----

This is true even if the garbage collector is called explicitly:
----
import gc
import ctypes
def hi():
    class c1(ctypes.Structure):
            _fields_=[('f1',ctypes.c_uint8)]
    class c2(ctypes.Structure):
            _fields_=[('g1',c1*2)]

while True:
    test=hi()
    test2=gc.collect()
----

It is also true if "del" is called on the return value of the function.

In order to get the memory leak, it appears to be necessary to have two Structure classes, with one of them containing a "multiple" of the other one as a field.

In a code project where functions returning Structure classes similarly to this example are being used, MemoryError has been encountered.

See: http://stackoverflow.com/users/553702/user553702
msg144171 - (view) Author: M (a01) Date: 2011-09-17 06:30
Correction to the above link: See: http://stackoverflow.com/questions/7452625/python-ctypes-memory-leak-with-structure
msg144172 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2011-09-17 06:41
This is caused by a cache which is kept of array's for different (Structure, length) pairs.
msg144173 - (view) Author: M (a01) Date: 2011-09-17 06:42
If it is a cache, shouldn't it be garbage-collected or limited in size? Why does it grow without bounds?
msg144178 - (view) Author: (lunaryorn) Date: 2011-09-17 08:05
Why should it?  After all, you're sort of abusing ctypes by repeatedly creating Struture types over and over again.  C structures that you might want to wrap with these types are fixed and known at the time of programming, so there is never a need to create the same Structure type twice.  Thus the set of Structure subclasses created during the live-time of a program is both, fixed and small, so there is no need to limit the cache size.
msg144179 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-09-17 09:27
I can reproduce the leak with Python 2.5.4, but not with Python 2.6.5
or Python 3.2.

Python 2.5.4 is an ancient version. Please upgrade to Python 2.7
or Python 3.2. If the leak still exists, just respond to this issue
and it will be opened again automatically.
History
Date User Action Args
2011-11-03 18:09:57skrahsetstatus: pending -> closed
2011-09-17 09:27:06skrahsetstatus: open -> pending

nosy: + skrah
messages: + msg144179

resolution: out of date
stage: resolved
2011-09-17 08:05:21lunaryornsetnosy: + lunaryorn
messages: + msg144178
2011-09-17 06:42:43a01setmessages: + msg144173
2011-09-17 06:41:23alexsetnosy: + alex
messages: + msg144172
2011-09-17 06:30:57a01setmessages: + msg144171
2011-09-17 06:29:26a01create