classification
Title: ctypes: unions as arguments
Type: crash Stage: resolved
Components: ctypes, Documentation Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: vinay.sajip Nosy List: Arfrever, alex, ammar2, arigo, docs@python, eli.bendersky, meador.inge, python-dev, theller, vinay.sajip, vstinner
Priority: normal Keywords: patch

Created on 2012-11-29 04:24 by arigo, last changed 2019-11-08 08:14 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
test184_lib.tgz arigo, 2012-11-29 04:24
Pull Requests
URL Status Linked Edit
PR 15839 vinay.sajip, 2019-09-10 11:38
PR 16430 closed vinay.sajip, 2019-09-26 16:48
PR 16799 merged vinay.sajip, 2019-10-15 08:36
PR 17016 merged vinay.sajip, 2019-10-31 13:01
PR 17017 merged vinay.sajip, 2019-10-31 13:09
PR 17064 merged ammar2, 2019-11-05 18:55
Messages (16)
msg176627 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2012-11-29 04:24
ctypes pretends to support passing arguments to C functions that are unions (not pointers to unions), but that's a lie.  In fact, the underlying libffi does not support it.  The attached example misbehaves on Linux x86-64.
msg183461 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2013-03-04 14:54
A minimal fix would be to update the documentation.

A more comprehensive fix would be to tweak ctypes to reject unions and bit-fields when running on non-x86 (does this work for ARM and other non-Intel archs?)

An even more comprehensive fix would be to make this work, but that would require forking libffi and should presumably be first implemented upstream.
msg183464 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2013-03-04 15:12
See also http://bugs.python.org/issue16576.
msg183817 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-03-09 13:56
New changeset 0acd9408b6f1 by Eli Bendersky in branch '3.2':
Add warning in ctypes documentation for #16575 and #16576
http://hg.python.org/cpython/rev/0acd9408b6f1

New changeset bfc159f8e4b4 by Eli Bendersky in branch '3.3':
Add warning in ctypes documentation for #16575 and #16576
http://hg.python.org/cpython/rev/bfc159f8e4b4

New changeset 502624235c7b by Eli Bendersky in branch 'default':
Add warning in ctypes documentation for #16575 and #16576
http://hg.python.org/cpython/rev/502624235c7b
msg183819 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-03-09 13:57
New changeset eece32440a52 by Eli Bendersky in branch '2.7':
Add warning in ctypes documentation for #16575 and #16576
http://hg.python.org/cpython/rev/eece32440a52
msg186689 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2013-04-13 03:12
I've opened a libffi issue in an attempt to get this fixed upstream:

  https://github.com/atgreen/libffi/issues/33
msg351454 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-09-09 13:11
Link to issue has changed to:

https://github.com/libffi/libffi/issues/33
msg355741 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-31 08:04
New changeset 79d4ed102a5069c6cebaed2627cb1645637f0429 by Vinay Sajip in branch 'master':
bpo-16575: Add checks for unions passed by value to functions. (GH-16799)
https://github.com/python/cpython/commit/79d4ed102a5069c6cebaed2627cb1645637f0429
msg355751 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-31 13:34
New changeset 952899704800a5aa438e84d50f0b6fc498c72e89 by Vinay Sajip in branch '3.8':
[3.8] bpo-16575: Add checks for unions passed by value to functions. (GH-16799) (GH-17016)
https://github.com/python/cpython/commit/952899704800a5aa438e84d50f0b6fc498c72e89
msg355754 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-31 14:15
New changeset 0118d109d54bf75c99a8b0fa9aeae1a478ac4b7e by Vinay Sajip in branch '3.7':
[3.7] bpo-16575: Add checks for unions passed by value to functions. (GH-16799) (GH-17017)
https://github.com/python/cpython/commit/0118d109d54bf75c99a8b0fa9aeae1a478ac4b7e
msg355993 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-05 01:29
This change introduced a reference leak on Windows. Example on 3.7:

https://buildbot.python.org/all/#/builders/132/builds/645

test_ctypes leaked [174, 174, 174] references, sum=522
test_ctypes leaked [76, 77, 77] memory blocks, sum=230
msg355994 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-05 01:31
Same issue on x86 Gentoo Refleaks 3.7:

test_ctypes leaked [174, 174, 174] references, sum=522
test_ctypes leaked [76, 78, 76] memory blocks, sum=230

https://buildbot.python.org/all/#/builders/114/builds/631
msg356053 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2019-11-05 18:58
Opened https://github.com/python/cpython/pull/17064 to fix this. Essentially it's a tiny little oversight in the back-porting. In the 3.7 branch, we perform an attribute lookup for `from_param` before the union checking code, so we must remember to DECREF it. This is unlike master where the attribute lookup happens after the union checking code.



λ win32\python_d.exe -m test -R 3:3 test_ctypes
0:00:00 Run tests sequentially
0:00:00 [1/1] test_ctypes
beginning 6 repetitions
123456
......

== Tests result: SUCCESS ==

1 test OK.

Total duration: 14.8 sec
Tests result: SUCCESS
msg356141 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-11-06 15:40
New changeset 484edbf9bf1a9e6bae0fcb10a0c165b89ea79295 by Vinay Sajip (Ammar Askar) in branch '3.7':
bpo-16575: Fix refleak on passing unions in ctypes (GH-17064)
https://github.com/python/cpython/commit/484edbf9bf1a9e6bae0fcb10a0c165b89ea79295
msg356147 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2019-11-06 17:35
Will close after https://buildbot.python.org/all/#/builders?tags=%2Brefleak&tags=%2B3.7 go back to green.
msg356230 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-11-08 08:14
3.7 refleaks buildbots pass again (ignoring a few warnings).
History
Date User Action Args
2019-11-08 08:14:56vstinnersetresolution: fixed
messages: + msg356230
2019-11-07 02:39:27ammar2setstatus: open -> closed
stage: patch review -> resolved
2019-11-06 17:35:07ammar2setmessages: + msg356147
2019-11-06 15:40:14vinay.sajipsetmessages: + msg356141
2019-11-05 18:58:13ammar2setnosy: + ammar2
messages: + msg356053
2019-11-05 18:55:28ammar2setstage: resolved -> patch review
pull_requests: + pull_request16572
2019-11-05 01:31:11vstinnersetmessages: + msg355994
2019-11-05 01:29:42vstinnersetstatus: closed -> open

nosy: + vstinner
messages: + msg355993

resolution: fixed -> (no value)
2019-10-31 14:17:04vinay.sajipsetstatus: open -> closed
assignee: docs@python -> vinay.sajip
resolution: fixed
stage: patch review -> resolved
2019-10-31 14:15:18vinay.sajipsetmessages: + msg355754
2019-10-31 13:34:11vinay.sajipsetmessages: + msg355751
2019-10-31 13:09:54vinay.sajipsetpull_requests: + pull_request16536
2019-10-31 13:01:34vinay.sajipsetpull_requests: + pull_request16535
2019-10-31 08:04:02vinay.sajipsetmessages: + msg355741
2019-10-15 08:36:48vinay.sajipsetpull_requests: + pull_request16355
2019-09-26 16:48:13vinay.sajipsetpull_requests: + pull_request16009
2019-09-10 11:38:40vinay.sajipsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request15485
2019-09-09 13:11:43vinay.sajipsetmessages: + msg351454
2017-02-23 08:18:33vinay.sajipsetnosy: + vinay.sajip
2016-06-23 09:30:48berker.peksaglinkissue14354 superseder
2013-04-13 03:12:03eli.benderskysetmessages: + msg186689
2013-03-18 12:26:28eli.benderskysetnosy: + theller
2013-03-09 13:57:30python-devsetmessages: + msg183819
2013-03-09 13:56:20python-devsetnosy: + python-dev
messages: + msg183817
2013-03-04 15:12:57arigosetmessages: + msg183464
2013-03-04 14:54:53eli.benderskysetassignee: docs@python
components: + Documentation
versions: + Python 3.3, Python 3.4
nosy: + eli.bendersky, docs@python

messages: + msg183461
stage: needs patch
2012-12-14 08:30:56Arfreversetnosy: + Arfrever
2012-12-11 06:20:05meador.ingesetnosy: + meador.inge
2012-11-29 04:49:13arigosettype: crash
2012-11-29 04:33:04alexsetnosy: + alex
2012-11-29 04:24:04arigocreate