This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Reference cycle when importing ctypes
Type: resource usage Stage:
Components: ctypes Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, eryksun, iritkatriel, meador.inge, poq, python-dev, terry.reedy, vladris
Priority: normal Keywords: patch

Created on 2011-05-21 22:39 by poq, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
ctypes-leak.patch poq, 2012-02-03 20:10 review
Messages (11)
msg136485 - (view) Author: (poq) Date: 2011-05-21 22:39
When importing ctypes after gc.set_debug(gc.DEBUG_LEAK), the garbage collector finds a 'c_int_Array_3' class and some related objects.

The class is created in ctypes/_endian.py:
_array_type = type(c_int * 3)

It seems that this could be avoided with:
_array_type = type(Array)

Of course, I realize this is not a bug because normally it will just get collected. It is just an extremely minor annoyance because this is currently the only thing still found by DEBUG_LEAK for my program ;)
msg137148 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-28 19:50
If you are able to rebuild Python, have you tried running the ctypes test after rebuilding with this change? And, does the test cover the internal uses of _array_type?
msg137371 - (view) Author: (poq) Date: 2011-05-31 17:07
Tests succeed with this change.

There is only one use of _array_type, which is in the same module. This use is presumably tested, because the test fails if I change the line to _array_type = type(Structure).

In fact, everything must behave exactly the same after this change, because the two values are identical:

>>> from ctypes import *
>>> type(c_int * 3) is type(Array)
True
msg137379 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-05-31 18:25
Thank you for the test and explanation. There currently is no specific cytpes maintainer. But from what you have said, I might feel comfortable enough applying this, if no one else does, when I have the necessary setup on a new machine.
msg140098 - (view) Author: Vlad Riscutia (vladris) Date: 2011-07-11 04:16
I ran full test suit after making the _array_type = type(Array) change and everything passes.

I also took a look at this and found additional leak. gc shows this as garbage:

[(<class '_ctypes._SimpleCData'>,), <class 'ctypes.c_longdouble'>, <attribute '_
_dict__' of 'c_longdouble' objects>, <attribute '__weakref__' of 'c_longdouble'
objects>, (<class 'ctypes.c_longdouble'>, <class '_ctypes._SimpleCData'>, <class
 '_ctypes._CData'>, <class 'object'>), {'__dict__': <attribute '__dict__' of 'c_
longdouble' objects>, '_type_': 'g', '__module__': 'ctypes', '__weakref__': <att
ribute '__weakref__' of 'c_longdouble' objects>, '__doc__': None}]

This is all caused by these lines in ctypes __init__.py:

class c_longdouble(_SimpleCData):
    _type_ = "g"
if sizeof(c_longdouble) == sizeof(c_double):
    c_longdouble = c_double

For me sizeof(c_longdouble) == sizeof(c_double) (I believe MS compiler always does this) but when we assign c_longdouble = c_double, there is a leak. I removed the alias lines:

if sizeof(c_longdouble) == sizeof(c_double):
    c_longdouble = c_double

And the leak was gone. Looks like changing c_longdouble after declaring it causes a leak. Below for similar aliasing of longlong types, we have this:

if _calcsize("l") == _calcsize("q"):
    # if long and long long have the same size, make c_longlong an alias for c_long
    c_longlong = c_long
    c_ulonglong = c_ulong
else:
    class c_longlong(_SimpleCData):
        _type_ = "q"
    _check_size(c_longlong)

    class c_ulonglong(_SimpleCData):
        _type_ = "Q"

This avoids declaring c_longlong and c_ulonglong as class if not needed to. The problem is _calcsize("g") causes an error because "g" is used as long double througout ctypes but _calcsize is function from _struct.c, where "g" (long double) is not defined. Not sure why it isn't...

So in short:
As far as I can tell _array_type = type(Array) doesn't break anything
Looks like we have another leak in ctypes (which isn't a big deal)
We have elegant fix for the leak once _struct.c will support long double
msg152552 - (view) Author: (poq) Date: 2012-02-03 20:10
I've attached a patch for the _array_type change.

The long double fix is probably dependent on PEP3118 (#3132).
msg152661 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012-02-05 06:29
New changeset 205da7a19a78 by Meador Inge in branch '3.2':
Issue #12142: Fixed reference cycle when importing ctypes
http://hg.python.org/cpython/rev/205da7a19a78

New changeset b228d9da8bd3 by Meador Inge in branch 'default':
Issue #12142: Fixed reference cycle when importing ctypes
http://hg.python.org/cpython/rev/b228d9da8bd3

New changeset 7cdbf627f958 by Meador Inge in branch '2.7':
Issue #12142: Fixed reference cycle when importing ctypes
http://hg.python.org/cpython/rev/7cdbf627f958
msg164258 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-06-28 14:28
Meador, can we close this issue?
msg164262 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2012-06-28 14:53
> Meador, can we close this issue?

I wanted to keep it open until the 'long double' problem is fixed as well.
msg408111 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-09 12:17
Looks like the long double issue is still there in 3.11

>>> import gc
>>> gc.set_debug(gc.DEBUG_LEAK)
>>> import ctypes
gc: collectable <function 0x0000026417BBE200>
gc: collectable <tuple 0x0000026417BE0040>
gc: collectable <dict 0x0000026417BC56C0>
gc: collectable <type 0x0000026417AD9840>
gc: collectable <tuple 0x0000026417BD6540>
gc: collectable <getset_descriptor 0x0000026417BD6580>
gc: collectable <getset_descriptor 0x0000026417BD65C0>
gc: collectable <tuple 0x0000026417BE05E0>
gc: collectable <_ctypes.PyCSimpleType 0x0000026417AE4E10>
gc: collectable <tuple 0x0000026417BD96C0>
gc: collectable <getset_descriptor 0x0000026417BE9080>
gc: collectable <getset_descriptor 0x0000026417BE90C0>
gc: collectable <StgDict 0x00000264178A5490>
>>> gc.garbage
[<function _C._m at 0x0000026417BBE200>, (<class 'object'>,), {'__module__': 'types', '_m': <function _C._m at 0x0000026417BBE200>, '__dict__': <attribute '__dict__' of '_C' objects>, '__weakref__': <attribute '__weakref__' of '_C' objects>, '__doc__': None}, <class 'types._C'>, (<class 'types._C'>, <class 'object'>), <attribute '__dict__' of '_C' objects>, <attribute '__weakref__' of '_C' objects>, (<class '_ctypes._SimpleCData'>,), <class 'ctypes.c_longdouble'>, (<class 'ctypes.c_longdouble'>, <class '_ctypes._SimpleCData'>, <class '_ctypes._CData'>, <class 'object'>), <attribute '__dict__' of 'c_longdouble' objects>, <attribute '__weakref__' of 'c_longdouble' objects>, {'__module__': 'ctypes', '_type_': 'g', '__dict__': <attribute '__dict__' of 'c_longdouble' objects>, '__weakref__': <attribute '__weakref__' of 'c_longdouble' objects>, '__doc__': None}]
msg408121 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-12-09 13:37
The _ctypes extension module could have a dict that maps each format code to its (size, alignment), based on `formattable`. Then direct size comparisons wouldn't be limited to types defined by the struct module, and it wouldn't be necessary to create c_longdouble just to check its size and throw it away.
History
Date User Action Args
2022-04-11 14:57:17adminsetgithub: 56351
2021-12-09 13:37:57eryksunsetnosy: + eryksun
messages: + msg408121
2021-12-09 12:17:03iritkatrielsetnosy: + iritkatriel

messages: + msg408111
versions: + Python 3.11, - Python 3.3
2012-06-28 14:53:45meador.ingesetmessages: + msg164262
2012-06-28 14:28:43amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg164258
2012-02-05 06:29:10python-devsetnosy: + python-dev
messages: + msg152661
2012-02-03 20:20:01pitrousetnosy: + meador.inge
2012-02-03 20:10:36poqsetfiles: + ctypes-leak.patch
keywords: + patch
messages: + msg152552
2011-07-11 04:16:13vladrissetnosy: + vladris
messages: + msg140098
2011-05-31 18:25:20terry.reedysetmessages: + msg137379
2011-05-31 17:07:51poqsetmessages: + msg137371
2011-05-28 19:50:49terry.reedysetnosy: + terry.reedy
messages: + msg137148
2011-05-21 22:41:11poqsettitle: eference cycle when importing ctypes -> Reference cycle when importing ctypes
2011-05-21 22:41:04poqsettitle: Circular reference when importing ctypes -> eference cycle when importing ctypes
2011-05-21 22:39:48poqcreate