classification
Title: Crash when subclassing ctypes.Union
Type: crash Stage: resolved
Components: ctypes Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lukasz.langa, ned.deily, oriordan, steve.dower, vinay.sajip, xtreak
Priority: release blocker Keywords: 3.7regression, 3.8regression, patch

Created on 2019-10-04 08:36 by oriordan, last changed 2019-10-15 07:44 by ned.deily. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16589 merged vinay.sajip, 2019-10-04 23:43
PR 16671 merged vinay.sajip, 2019-10-09 05:17
PR 16672 merged vinay.sajip, 2019-10-09 05:31
Messages (12)
msg353906 - (view) Author: Doug O'Riordan (oriordan) Date: 2019-10-04 08:36
Ran into Segfaults while trying to use pysnmp with 3.8.0rc1.
The code is running fine on 3.8.0b04.

$ python3.8
Python 3.8.0rc1 (default, Oct  2 2019, 14:15:18)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> class in6_addr_U(ctypes.Union):
...     _fields_ = [
...         ('__u6_addr8', ctypes.c_uint8 * 16),
...         ('__u6_addr16', ctypes.c_uint16 * 8),
...         ('__u6_addr32', ctypes.c_uint32 * 4),
...     ]
...
Segmentation fault

$ docker run -it python:3.8.0rc1-slim
Python 3.8.0rc1 (default, Oct  2 2019, 23:38:42)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> class in6_addr_U(ctypes.Union):
...     _fields_ = [
...         ('__u6_addr8', ctypes.c_uint8 * 16),
...         ('__u6_addr16', ctypes.c_uint16 * 8),
...         ('__u6_addr32', ctypes.c_uint32 * 4),
...     ]
...
$

The code is from here: https://github.com/etingof/pysnmp/blob/master/pysnmp/carrier/sockmsg.py#L47-L52
msg353909 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-10-04 08:55
I am adding 3.8 regression since the report says 3.8.0b4 works and segfaults with 3.8.0RC1.
msg353955 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-10-04 16:54
Also crashes on Windows (x64 and x86), and on 3.7.5rc1, so this is likely in our code.
msg353956 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-10-04 16:57
I get this assertion in a debug build:

Assertion failed: actual_type_index <= MAX_ELEMENTS, file c:\projects\cpython\modules\_ctypes\stgdict.c, line 718
msg353958 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-10-04 17:07
Increasing MAX_ELEMENTS fixes it, but I'm not sure what other impacts there are from doing that.

+Vinay who added the array handling that's hitting the limit.
msg353959 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-10-04 17:08
Given this limit can easily be hit by user code, I'd like to see it turned into a proper check with an exception (or a dynamic array) rather than just an assertion. We should not segfault here.

while (length > 0) {
    actual_types[actual_type_index++] = &edict->ffi_type_pointer;
    assert(actual_type_index <= MAX_ELEMENTS);
    length--;
}
msg353967 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-04 18:26
> We should not segfault here.

Agreed, MAX_ELEMENTS was set to be an upper bound which shouldn't be hit. I'll investigate with OP's example data and see where the bug is.
msg354233 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-08 20:59
New changeset e8bedbddadaa86be6bd86dc32dbdbd53933a4988 by Vinay Sajip in branch 'master':
bpo-38368: Added fix for ctypes crash when handling arrays in structs… (GH-16589)
https://github.com/python/cpython/commit/e8bedbddadaa86be6bd86dc32dbdbd53933a4988
msg354249 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-09 05:48
New changeset d004a5b082d0770682e6efbf03105a67469d4820 by Vinay Sajip in branch '3.8':
bpo-38368: Added fix for ctypes crash when handling arrays in structs/unions. (GH-16589) (GH-16671)
https://github.com/python/cpython/commit/d004a5b082d0770682e6efbf03105a67469d4820
msg354250 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2019-10-09 05:48
New changeset 129c2b3d44a962fc5964ef8fd4e6b22dbb691a6c by Vinay Sajip in branch '3.7':
bpo-38368: Added fix for ctypes crash when handling arrays in structs/unions. (GH-16589) (GH-16672)
https://github.com/python/cpython/commit/129c2b3d44a962fc5964ef8fd4e6b22dbb691a6c
msg354692 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-10-15 07:30
New changeset 1c61f2cc7ab44591b1832307a4022c927502d446 by Ned Deily (Vinay Sajip) in branch '3.7':
bpo-38368: Added fix for ctypes crash when handling arrays in structs/unions. (GH-16589) (GH-16672)
https://github.com/python/cpython/commit/1c61f2cc7ab44591b1832307a4022c927502d446
msg354704 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-10-15 07:44
(fix released in 3.8.0 and 3.7.5)
History
Date User Action Args
2019-10-15 07:44:07ned.deilysetmessages: + msg354704
2019-10-15 07:30:24ned.deilysetmessages: + msg354692
2019-10-09 05:53:36vinay.sajipsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-10-09 05:48:50vinay.sajipsetmessages: + msg354250
2019-10-09 05:48:01vinay.sajipsetmessages: + msg354249
2019-10-09 05:31:00vinay.sajipsetpull_requests: + pull_request16253
2019-10-09 05:17:20vinay.sajipsetkeywords: + patch
pull_requests: + pull_request16252
2019-10-08 20:59:08vinay.sajipsetmessages: + msg354233
2019-10-08 11:04:03dokosetkeywords: + 3.7regression, - patch
2019-10-04 23:43:22vinay.sajipsetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request16179
2019-10-04 18:26:45vinay.sajipsetmessages: + msg353967
2019-10-04 17:08:46steve.dowersetmessages: + msg353959
2019-10-04 17:07:01steve.dowersetnosy: + vinay.sajip

messages: + msg353958
versions: + Python 3.9
2019-10-04 16:57:51steve.dowersetmessages: + msg353956
2019-10-04 16:54:52steve.dowersetpriority: normal -> release blocker
versions: + Python 3.7
nosy: + steve.dower, ned.deily

messages: + msg353955

stage: test needed
2019-10-04 08:55:46xtreaksetkeywords: + 3.8regression
nosy: + xtreak, lukasz.langa
messages: + msg353909

2019-10-04 08:36:01oriordancreate