classification
Title: ctypes c_long & c_bool have incorrect PEP-3118 type codes
Type: behavior Stage: resolved
Components: ctypes Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, belopolsky, csabella, mark.dickinson, meador.inge, pitrou, pv, theller
Priority: normal Keywords: patch

Created on 2010-12-20 23:37 by pv, last changed 2017-09-02 16:24 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
001-ctypes-fix-pep-3118-type-codes-for-c-long-and-c-bool.patch pv, 2010-12-20 23:37
Pull Requests
URL Status Linked Edit
PR 31 merged pv, 2017-02-14 19:24
PR 3241 merged pv, 2017-08-30 08:14
PR 3242 merged pv, 2017-08-30 08:32
Messages (10)
msg124411 - (view) Author: Pauli Virtanen (pv) * Date: 2010-12-20 23:37
Currently on Python 3.x:

>>> import ctypes
>>> memoryview(ctypes.c_long()).format
'<l'

This is invalid on 64-bit platforms: the above means 32-bit little-endian float. The '<' endian specification turns on the "standard size" mode (similarly as for the struct module), which makes type character have a platform-independent meaning.

Unfortunately, the struct module format syntax does *not* allow specifying native-size non-native-endian items. So just replacing '<' by '^' cannot be in general be done.

Suggested fix attached. It adds a converter function that maps the platform-dependent ctypes codes to struct module standard-size codes; handling c_long and c_bool specially.

    ***

After this patch (and the one in http://bugs.python.org/issue10744 ):

>>> import numpy as np
>>> from ctypes import *
>>> class Point(Structure):
...     _fields_ = [("x", c_long), ("y", c_long)]
... 
>>> class StructWithArrays(Structure):
...     _fields_ = [("x", c_long * 3 * 2), ("y", Point * 4)]
... 
>>> x = StructWithArrays()
>>> y = np.asarray(x)
>>> y.dtype
dtype([('x', '<i8', (2, 3)), ('y', [('x', '<i8'), ('y', '<i8')], (4,))])
>>> y['x'] = [[1,2,3],[4,5,6]]
>>> y['y']['x'] = np.arange(4) + 10
>>> y['y']['y'] = np.arange(4) + 20
>>> x.x[0][0]
1
>>> x.x[0][1]
2
>>> x.x[0][2]
3
>>> x.y[0].x
10
>>> x.y[1].x
11
>>> x.y[0].y
20
>>> x.y[1].y
21
msg220819 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-17 12:54
Could someone do a patch review on this please.
msg287795 - (view) Author: Pauli Virtanen (pv) * Date: 2017-02-14 19:24
Converted patch to Github PR + some cleanup.
msg295247 - (view) Author: Cheryl Sabella (csabella) * Date: 2017-06-06 11:13
Added the names from the experts list to this ticket.
msg300951 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-28 12:08
New changeset 07f1658aa09f6798793c473c72b2951b7fefe220 by Antoine Pitrou (Pauli Virtanen) in branch 'master':
bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (#31)
https://github.com/python/cpython/commit/07f1658aa09f6798793c473c72b2951b7fefe220
msg300952 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-28 12:13
Pauli, do you want to submit backport PRs? This can be done using the cherry_picker script: https://devguide.python.org/committing/?highlight=cherry_picker#backporting-changes-to-an-older-version
msg301015 - (view) Author: Pauli Virtanen (pv) * Date: 2017-08-30 08:16
Created backport PR for 3.6: https://github.com/python/cpython/pull/3241
Which versions take backports?
msg301016 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-30 08:17
At this point, 2.7 and 3.6 do.
msg301019 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-30 09:40
New changeset 2d1653aa43cf02e6b74f9d4f178fac9969a293e2 by Antoine Pitrou (Pauli Virtanen) in branch '3.6':
[3.6] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) (#3241)
https://github.com/python/cpython/commit/2d1653aa43cf02e6b74f9d4f178fac9969a293e2
msg301165 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-09-02 16:24
New changeset 990b2d043cdfaafc3378550f7d27259410a6b918 by Antoine Pitrou (Pauli Virtanen) in branch '2.7':
[2.7] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31) (#3242)
https://github.com/python/cpython/commit/990b2d043cdfaafc3378550f7d27259410a6b918
History
Date User Action Args
2017-09-02 16:24:58pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-09-02 16:24:37pitrousetmessages: + msg301165
2017-08-30 09:40:08pitrousetmessages: + msg301019
2017-08-30 08:32:48pvsetpull_requests: + pull_request3286
2017-08-30 08:17:05pitrousetmessages: + msg301016
2017-08-30 08:16:23pvsetmessages: + msg301015
2017-08-30 08:14:35pvsetpull_requests: + pull_request3284
2017-08-28 12:13:12pitrousetmessages: + msg300952
2017-08-28 12:08:53pitrousetmessages: + msg300951
2017-06-06 23:13:51pitrousetassignee: theller ->
stage: patch review
versions: + Python 2.7, Python 3.6, Python 3.7, - Python 3.1, Python 3.2, Python 3.3
2017-06-06 11:13:47csabellasetnosy: + csabella, belopolsky, amaury.forgeotdarc, meador.inge
messages: + msg295247
2017-02-14 20:15:28BreamoreBoysetnosy: - BreamoreBoy
2017-02-14 19:24:40pvsetmessages: + msg287795
pull_requests: + pull_request61
2014-10-14 16:45:25skrahsetnosy: - skrah
2014-06-17 12:54:32BreamoreBoysetnosy: + BreamoreBoy
messages: + msg220819
2011-08-10 12:43:43skrahsetnosy: + skrah
2010-12-21 19:25:35pitrousetnosy: + mark.dickinson
2010-12-21 19:20:51georg.brandlsetnosy: + pitrou
2010-12-20 23:37:52pvsetversions: + Python 3.1, Python 3.2, Python 3.3
nosy: + theller

assignee: theller
components: + ctypes
type: behavior
2010-12-20 23:37:20pvcreate