This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ctypes: better support of bytearray objects
Type: enhancement Stage: patch review
Components: ctypes Versions: Python 3.5
process
Status: open Resolution:
Dependencies: 22161 Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, belopolsky, martin.panter, meador.inge, mfxmfx, pitrou, serhiy.storchaka, theller
Priority: normal Keywords: needs review, patch

Created on 2011-01-02 06:59 by mfxmfx, last changed 2022-04-11 14:57 by admin.

Files
File name Uploaded Description Edit
ctype_bytearray.patch serhiy.storchaka, 2013-12-01 10:44 review
ctypes_bytearray_2.patch serhiy.storchaka, 2014-08-09 07:00 review
Messages (6)
msg125032 - (view) Author: Markus F.X.J. Oberhumer (mfxmfx) Date: 2011-01-02 06:59
Python 3.2b2 does not properly support accessing bytearrays from
ctypes, which makes dealing with large buffers somewhat unpleasant.

A very first fix - a simple patch for the z_set() function - is given here.


build/Python-3.2b2 $ quilt diff
Index: b/Modules/_ctypes/cfield.c
===================================================================
--- a/Modules/_ctypes/cfield.c
+++ b/Modules/_ctypes/cfield.c
@@ -1363,6 +1363,10 @@
         *(char **)ptr = PyBytes_AsString(value);
         Py_INCREF(value);
         return value;
+    } else if (PyByteArray_Check(value)) {
+        *(char **)ptr = PyByteArray_AsString(value);
+        Py_INCREF(value);
+        return value;
     } else if (PyLong_Check(value)) {
 #if SIZEOF_VOID_P == SIZEOF_LONG_LONG
         *(char **)ptr = (char *)PyLong_AsUnsignedLongLongMask(value);
msg204905 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-01 10:44
Here is preliminary patch which makes bytearray support in ctypes better.
msg224955 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-06 18:02
Why not use the buffer API instead?
msg224961 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-06 18:59
Because it is simpler and more efficient. The buffer API requires the releasing of buffer. Correct tracking the ownership of the buffer requires larger and more complicated patch. Even if support for general buffers will be added, I suppose it will be worth to left specialized paths for bytes and bytearrays in some places.
msg224997 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-07 08:33
Here are things which support bytes instances only:

1. Constructor and setter of the "value" attribute of NUL terminated char 
buffer.

>>> p = create_string_buffer(b"Hello")
>>> p.value
b'Hello'
>>> p.raw
b'Hello\x00'
>>> p.value = b'Bye'
>>> p.value
b'Bye'
>>> p.raw
b'Bye\x00o\x00'
>>> create_string_buffer(bytearray(b"Hello"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython-3.4/Lib/ctypes/__init__.py", line 63, in 
create_string_buffer
    raise TypeError(init)
TypeError: bytearray(b'Hello')
>>> p.value = bytearray(b'Hi')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: bytes expected instead of bytearray instance

But setter of the "raw" attribute accepts arbitrary bytes-like objects.

>>> p.raw = bytearray(b'Hi')
>>> p.raw
b'Hie\x00o\x00'

The patch adds support of bytearray here. It would be not so easy to add 
support for arbitrary bytes-like objects, and due to NUL-terminating it can be 
confused. I even not sure that support of bytearray is needed here.

2. Constructor of NUL terminated wchar buffer (create_unicode_buffer). Actually 
this doesn't work. Bytes argument is accepted, but then rejected in internal 
setting function. This is a bug, bytes should be removed here.

3. c_wchar_p.from_param() accepts bytes argument, but then reject it in 
internal function. This is a bug, bytes should be removed here.

4. c_void_p.from_param() accepts bytes and bytearray arguments, but then 
reject bytearray in internal function. This is a bug, either bytearray should 
be rejected here, or support of bytearray should be added in internal function 
(very easy, the patch does this). Adding support of arbitrary bytes-like 
objects is more complicated.

5. c_char_p.from_param() accepts bytes argument. Adding support for bytearray 
or arbitrary bytes-like objects has same complexity as in 
c_void_p.from_param().

6. Bytes arguments of call_function(), call_cdeclfunction() and 
CopyComPointer() are implicitly converted to pointers. It is easy to add 
support of bytearray, and more complicated for arbitrary bytes-like objects.
msg225101 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-09 07:00
Existing inconsistency was fixed in issue22161.

Patch updated, synchronized with tip. Added new tests.
History
Date User Action Args
2022-04-11 14:57:10adminsetgithub: 55012
2014-10-14 17:17:00skrahsetnosy: - skrah
2014-08-09 07:00:35serhiy.storchakasetfiles: + ctypes_bytearray_2.patch

messages: + msg225101
2014-08-07 09:13:36serhiy.storchakasetdependencies: + Remove unsupported code from ctypes
2014-08-07 08:33:52serhiy.storchakasetmessages: + msg224997
2014-08-06 18:59:31serhiy.storchakasetmessages: + msg224961
2014-08-06 18:02:12pitrousetassignee: theller ->

messages: + msg224955
nosy: + pitrou, skrah
2014-08-06 16:04:34serhiy.storchakasetkeywords: + needs review
2014-08-01 05:15:01martin.pantersetnosy: + martin.panter
2013-12-01 10:44:47serhiy.storchakasetfiles: + ctype_bytearray.patch

versions: + Python 3.5, - Python 3.2
keywords: + patch
nosy: + belopolsky, amaury.forgeotdarc, meador.inge, serhiy.storchaka

messages: + msg204905
stage: patch review
2011-01-02 06:59:01mfxmfxcreate