ctypes: unions as arguments #60779

arigo · 2012-11-29T04:24:04Z

BPO	16575
Nosy	@arigo, @theller, @vsajip, @vstinner, @alex, @meadori, @ammaraskar
PRs	bpo-22273: Update ctypes to correctly handle arrays in small structur… #15839 bpo-16575: Add checks for unions passed by value to functions. #16430 bpo-16575: Add checks for unions passed by value to functions. #16799 [3.8] bpo-16575: Add checks for unions passed by value to functions. … #17016 [3.7] bpo-16575: Add checks for unions passed by value to functions. (GH-16799) #17017 [3.7] bpo-16575: Fix refleak on passing unions in ctypes #17064 bpo-16575: Disabled checks for union types being passed by value. #17960 [3.8] bpo-16575: Disabled checks for union types being passed by value. (GH-17960) #17964 [3.7] bpo-16575: Disabled checks for union types being passed by value. (GH-17960) #17970
Files	test184_lib.tgz

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vsajip'
closed_at = None
created_at = <Date 2012-11-29.04:24:04.013>
labels = ['ctypes', 'type-crash', 'docs']
title = 'ctypes: unions as arguments'
updated_at = <Date 2020-01-12.20:55:57.455>
user = 'https://github.com/arigo'

bugs.python.org fields:

activity = <Date 2020-01-12.20:55:57.455>
actor = 'vinay.sajip'
assignee = 'vinay.sajip'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'ctypes']
creation = <Date 2012-11-29.04:24:04.013>
creator = 'arigo'
dependencies = []
files = ['28155']
hgrepos = []
issue_num = 16575
keywords = ['patch']
message_count = 20.0
messages = ['176627', '183461', '183464', '183817', '183819', '186689', '351454', '355741', '355751', '355754', '355993', '355994', '356053', '356141', '356147', '356230', '359834', '359836', '359844', '359876']
nosy_count = 11.0
nosy_names = ['arigo', 'theller', 'vinay.sajip', 'vstinner', 'Arfrever', 'alex', 'eli.bendersky', 'meador.inge', 'docs@python', 'python-dev', 'ammar2']
pr_nums = ['15839', '16430', '16799', '17016', '17017', '17064', '17960', '17964', '17970']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue16575'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

arigo · 2012-11-29T04:24:03Z

ctypes pretends to support passing arguments to C functions that are unions (not pointers to unions), but that's a lie. In fact, the underlying libffi does not support it. The attached example misbehaves on Linux x86-64.

elibendersky · 2013-03-04T14:54:53Z

A minimal fix would be to update the documentation.

A more comprehensive fix would be to tweak ctypes to reject unions and bit-fields when running on non-x86 (does this work for ARM and other non-Intel archs?)

An even more comprehensive fix would be to make this work, but that would require forking libffi and should presumably be first implemented upstream.

arigo · 2013-03-04T15:12:58Z

See also http://bugs.python.org/issue16576.

python-dev · 2013-03-09T13:56:20Z

New changeset 0acd9408b6f1 by Eli Bendersky in branch '3.2':
Add warning in ctypes documentation for bpo-16575 and bpo-16576
http://hg.python.org/cpython/rev/0acd9408b6f1

New changeset bfc159f8e4b4 by Eli Bendersky in branch '3.3':
Add warning in ctypes documentation for bpo-16575 and bpo-16576
http://hg.python.org/cpython/rev/bfc159f8e4b4

New changeset 502624235c7b by Eli Bendersky in branch 'default':
Add warning in ctypes documentation for bpo-16575 and bpo-16576
http://hg.python.org/cpython/rev/502624235c7b

python-dev · 2013-03-09T13:57:30Z

New changeset eece32440a52 by Eli Bendersky in branch '2.7':
Add warning in ctypes documentation for bpo-16575 and bpo-16576
http://hg.python.org/cpython/rev/eece32440a52

elibendersky · 2013-04-13T03:12:03Z

I've opened a libffi issue in an attempt to get this fixed upstream:

https://github.com/atgreen/libffi/issues/33

vsajip · 2019-09-09T13:11:44Z

Link to issue has changed to:

libffi/libffi#33

vsajip · 2019-10-31T08:04:02Z

New changeset 79d4ed1 by Vinay Sajip in branch 'master':
bpo-16575: Add checks for unions passed by value to functions. (GH-16799)
79d4ed1

vsajip · 2019-10-31T13:34:11Z

New changeset 9528997 by Vinay Sajip in branch '3.8':
[3.8] bpo-16575: Add checks for unions passed by value to functions. (GH-16799) (GH-17016)
9528997

vsajip · 2019-10-31T14:15:18Z

New changeset 0118d10 by Vinay Sajip in branch '3.7':
[3.7] bpo-16575: Add checks for unions passed by value to functions. (GH-16799) (GH-17017)
0118d10

vstinner · 2019-11-05T01:29:42Z

This change introduced a reference leak on Windows. Example on 3.7:

https://buildbot.python.org/all/#/builders/132/builds/645

test_ctypes leaked [174, 174, 174] references, sum=522
test_ctypes leaked [76, 77, 77] memory blocks, sum=230

vstinner · 2019-11-05T01:31:11Z

Same issue on x86 Gentoo Refleaks 3.7:

test_ctypes leaked [174, 174, 174] references, sum=522
test_ctypes leaked [76, 78, 76] memory blocks, sum=230

https://buildbot.python.org/all/#/builders/114/builds/631

ammaraskar · 2019-11-05T18:58:13Z

Opened #17064 to fix this. Essentially it's a tiny little oversight in the back-porting. In the 3.7 branch, we perform an attribute lookup for from_param before the union checking code, so we must remember to DECREF it. This is unlike master where the attribute lookup happens after the union checking code.

λ win32\python_d.exe -m test -R 3:3 test_ctypes
0:00:00 Run tests sequentially
0:00:00 [1/1] test_ctypes
beginning 6 repetitions
123456
......

== Tests result: SUCCESS ==

1 test OK.

Total duration: 14.8 sec
Tests result: SUCCESS

vsajip · 2019-11-06T15:40:14Z

New changeset 484edbf by Vinay Sajip (Ammar Askar) in branch '3.7':
bpo-16575: Fix refleak on passing unions in ctypes (GH-17064)
484edbf

ammaraskar · 2019-11-06T17:35:07Z

Will close after https://buildbot.python.org/all/#/builders?tags=%2Brefleak&tags=%2B3.7 go back to green.

vstinner · 2019-11-08T08:14:56Z

3.7 refleaks buildbots pass again (ignoring a few warnings).

vsajip · 2020-01-12T07:44:02Z

It seems that notwithstanding the issues identified in the original bug report, calls by value of structs containing unions are being used out there in the wild. Examples:

The comtypes library (Windows):
#16799 (comment)

The ctypesgen library, which is used by other projects across platforms:
ctypesgen/ctypesgen#77

Linux examples:
waveform80/picamera#604
hcpl/xkbgroup#15
enkore/i3pystatus#771

So, it would seem that the simplest course of action is to disable the checks for now. The checks will be commented out and this issue and the related issue bpo-16576 will be reopened.

vsajip · 2020-01-12T08:54:06Z

New changeset c12440c by Vinay Sajip in branch 'master':
bpo-16575: Disabled checks for union types being passed by value. (GH-17960)
c12440c

vsajip · 2020-01-12T11:41:11Z

New changeset eb9ba2f by Vinay Sajip (Miss Islington (bot)) in branch '3.8':
bpo-16575: Disabled checks for union types being passed by value. (GH-17960) (GH-17964)
eb9ba2f

vsajip · 2020-01-12T20:55:57Z

New changeset 9dbf5d3 by Vinay Sajip in branch '3.7':
[3.7] bpo-16575: Disabled checks for union types being passed by value. (GH-17960) (GH-17970)
9dbf5d3

arigo mannequin added topic-ctypes type-crash A hard crash of the interpreter, possibly with a core dump labels Nov 29, 2012

elibendersky mannequin added the docs Documentation in the Doc dir label Mar 4, 2013

elibendersky mannequin assigned docspython Mar 4, 2013

vsajip closed this as completed Oct 31, 2019

vsajip assigned vsajip and unassigned docspython Oct 31, 2019

vstinner reopened this Nov 5, 2019

ammaraskar closed this as completed Nov 7, 2019

vsajip reopened this Jan 12, 2020

ezio-melotti transferred this issue from another repository Apr 10, 2022

hauntsaninja mentioned this issue Oct 29, 2022

ctypes: structure with bitfields as argument #60780

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ctypes: unions as arguments #60779

ctypes: unions as arguments #60779

arigo mannequin commented Nov 29, 2012

arigo mannequin commented Nov 29, 2012

elibendersky mannequin commented Mar 4, 2013

arigo mannequin commented Mar 4, 2013

python-dev mannequin commented Mar 9, 2013

python-dev mannequin commented Mar 9, 2013

elibendersky mannequin commented Apr 13, 2013 •

edited by vsajip

vsajip commented Sep 9, 2019

vsajip commented Oct 31, 2019

vsajip commented Oct 31, 2019

vsajip commented Oct 31, 2019

vstinner commented Nov 5, 2019

vstinner commented Nov 5, 2019

ammaraskar commented Nov 5, 2019

vsajip commented Nov 6, 2019

ammaraskar commented Nov 6, 2019

vstinner commented Nov 8, 2019

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

ctypes: unions as arguments #60779

ctypes: unions as arguments #60779

Comments

arigo mannequin commented Nov 29, 2012

arigo mannequin commented Nov 29, 2012

elibendersky mannequin commented Mar 4, 2013

arigo mannequin commented Mar 4, 2013

python-dev mannequin commented Mar 9, 2013

python-dev mannequin commented Mar 9, 2013

elibendersky mannequin commented Apr 13, 2013 • edited by vsajip

vsajip commented Sep 9, 2019

vsajip commented Oct 31, 2019

vsajip commented Oct 31, 2019

vsajip commented Oct 31, 2019

vstinner commented Nov 5, 2019

vstinner commented Nov 5, 2019

ammaraskar commented Nov 5, 2019

vsajip commented Nov 6, 2019

ammaraskar commented Nov 6, 2019

vstinner commented Nov 8, 2019

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

vsajip commented Jan 12, 2020

elibendersky mannequin commented Apr 13, 2013 •

edited by vsajip