This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ctypes is highly eclectic in its raw-memory support
Type: enhancement Stage:
Components: ctypes Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: theller Nosy List: Rondom, amaury.forgeotdarc, belopolsky, benrg, eryksun, martin.panter, meador.inge, theller
Priority: normal Keywords:

Created on 2011-03-07 07:49 by benrg, last changed 2022-04-11 14:57 by admin.

Messages (5)
msg130236 - (view) Author: (benrg) Date: 2011-03-07 07:49
ctypes accepts bytes objects as arguments to C functions, but not bytearray objects. It has its own array types but seems to be unaware of array.array. It doesn't even understand memoryview objects. I think that all of these types should be passable to C code.

Additionally, while passing a pointer to a bytes value to a C function is easy, it's remarkably difficult to pass that same pointer with an offset added to it. I first tried byref(buf, offset), but byref wouldn't accept bytes. Then I tried addressof(buf), but that didn't work either, even though ctypes is clearly able to obtain this address when it has to. After banging my head against the wall for longer than I care to think about, I finally came up with something like byref((c_char*length).from_buffer(buf), offset). But that broke in 3.2. After wasting even more time, I came up with addressof(cast(buf, POINTER(c_char)).contents) + offset. This is nuts. There should be a simple and documented way to do this. My first preference would be for the byref method, since it was the first thing I tried, and would have saved me the most time. Ideally both byref and addressof should work for bytes objects as they do for ctypes arrays (and also for bytearray, memoryview, etc.)
msg220904 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-17 22:00
Your opinions please gentlemen.
msg224447 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-07-31 21:46
Extending byref to support bytes and str objects sounds reasonable. 

Here's another workaround to pass a bytes object with an offset:

    from ctypes import *
    from ctypes.util import find_library

    class offset:
        def __init__(self, arg, offset):
            self.arg = arg
            self.offset = offset

    class my_char_p(c_char_p):
        @classmethod
        def from_param(cls, arg):
            if isinstance(arg, offset):
                t = cast(arg.arg, POINTER(c_char * 0))[0]
                carg = byref(t, arg.offset)
            else:
                carg = super().from_param(arg)
            return carg

    atoi = CDLL(find_library('c')).atoi
    atoi.argtypes = [my_char_p]

    >>> atoi(b'12345')
    12345
    >>> atoi(offset(b'12345', 1))
    2345
    >>> atoi(offset(b'12345', 3))
    45

You can also convert bytearray, memoryview, and array.array objects from_param. If the object's buffer is writable you can use a ctypes type's from_buffer method to create the C arg. This takes an optional offset argument. Otherwise use from_buffer_copy or cast.
msg224455 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-07-31 23:29
Interesting that “cast” accepts a byte string. If this is intended behaviour, it would be good to document that. Currently it says it takes “an object that can be interpreted as a pointer”.
msg224466 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-08-01 04:13
> Interesting that “cast” accepts a byte string. If this is 
> intended behaviour, it would be good to document that. 
> Currently it says it takes “an object that can be 
> interpreted as a pointer”.

cast makes an FFI call:

    _cast = PYFUNCTYPE(py_object,
                       c_void_p, py_object, py_object)(_cast_addr)
    def cast(obj, typ):
        return _cast(obj, obj, typ)

The first arg is passed as c_void_p, i.e. a void pointer. c_void_p.from_param accepts common objects that can be interpreted as a pointer: None (NULL), integers, bytes, and str. It also accepts c_void_p ('P'), c_char_p ('z'), c_wchar_p ('Z'), Array, _Pointer, _CFuncPtr, and CArgObject (byref). 

If none of the latter apply, c_void_p.from_param checks for the _as_parameter_ hook. For example:

    from ctypes import *
    from ctypes.util import find_library

    libc = CDLL(find_library('c'))
    libc.atoi.argtypes = [c_void_p]
    
    class X: 
        _as_parameter_ = b'123'

    >>> libc.atoi(X())
    123

There's also code in place to support bytearray, but it's incomplete. It uses the z_set setfunc (defined in cfield.c), which doesn't support bytearray yet.
History
Date User Action Args
2022-04-11 14:57:14adminsetgithub: 55638
2021-03-12 20:49:24eryksunsetversions: + Python 3.8, Python 3.9, Python 3.10, - Python 3.5
2016-10-13 19:26:35BreamoreBoysetnosy: - BreamoreBoy
2016-10-11 15:49:07Rondomsetnosy: + Rondom
2014-08-01 04:13:07eryksunsetmessages: + msg224466
2014-07-31 23:29:36martin.pantersetmessages: + msg224455
2014-07-31 21:46:41eryksunsetnosy: + eryksun
messages: + msg224447
2014-07-31 06:32:23martin.pantersetnosy: + martin.panter
2014-06-17 22:00:57BreamoreBoysetnosy: + belopolsky, amaury.forgeotdarc, meador.inge, BreamoreBoy

messages: + msg220904
versions: + Python 3.5, - Python 3.2
2011-03-07 07:49:00benrgcreate