This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: c_void_p array is a footgun on I32LP64 systems
Type: behavior Stage:
Components: ctypes Versions: Python 3.11, Python 3.10, Python 3.9
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, taralx
Priority: normal Keywords:

Created on 2022-03-09 02:28 by taralx, last changed 2022-04-11 14:59 by admin.

Messages (3)
msg414783 - (view) Author: JP Sugarbroad (taralx) * Date: 2022-03-09 02:28
The following code will likely crash on I32LP64 systems:

dim = lib.get_array_size(opaque)
ptrs = (c_void_p * dim)()
lib.get_array_values(opaque, ptrs)
for ptr in ptrs:

What happens is that `ptr` is not a `c_void_p` -- it's just a bare number. And when it's passed to another function it goes in as a (32-bit) `c_int`, resulting in a truncation.

I'm not sure what can be done here (maybe a truncation warning?) but it's definitely a difficult bug to notice when reviewing code.
msg414790 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-03-09 10:14
> I'm not sure what can be done here (maybe a truncation warning?)

For a function pointer, the default argument conversion for Python integers is the platform int type. Ideally, Python integer arguments would be converted to a type that matches the platform word size, as is the default behavior for integer arguments in C. But the behavior can't be changed at this point. Ideally, it would behave the same in LP64 (Unix) and LLP64 (Windows) systems, but OverflowError is raised in LLP64 because ctypes first converts to a long int. OverflowError could be manually raised if `(unsigned long)value > UINT_MAX`, but I think it's also too late to make that change. Scripts have worked around the current behavior for about two decades. Raising a warning is really the best that could be done, if anything is done at all.

The best solution is to not use bare function pointers without setting the prototype. If a function pointer is created as an attribute of a CDLL instance, the common way to define the prototype is by setting the function's `argtypes` and `restype` attributes.

Another ctypes concept to be aware of is that subclasses of simple types do not get converted by default when accessed as C fields, array subscripts, or function results. For example:

    class my_void_p(ctypes.c_void_p):

    >>> a = (my_void_p * 1)()
    >>> isinstance(a[0], my_void_p)
msg414805 - (view) Author: JP Sugarbroad (taralx) * Date: 2022-03-09 16:47
That matches our expectation. A subclass works - perhaps `c_void_p` could be deprecated in favor of a built-in subclass for generic opaque pointers as well?

Glad you agree that a warning would be useful here.
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91122
2022-03-09 16:47:23taralxsetmessages: + msg414805
2022-03-09 10:14:42eryksunsetnosy: + eryksun

messages: + msg414790
versions: - Python 3.7, Python 3.8
2022-03-09 02:28:03taralxcreate