Title: ctypes cast and from_address cause crash on Windows 10
Type: crash Stage: resolved
Components: ctypes Versions: Python 3.7, Python 3.6, Python 3.5
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Mordechai Botrashvily, eryksun
Priority: normal Keywords:

Created on 2018-11-27 11:00 by Mordechai Botrashvily, last changed 2018-11-28 17:26 by Mordechai Botrashvily. This issue is now closed.

File name Uploaded Description Edit Mordechai Botrashvily, 2018-11-27 11:00 Code to reproduce the crash on Windows 10 1803 and python 3.6.2
Messages (3)
msg330503 - (view) Author: Mordechai Botrashvily (Mordechai Botrashvily) Date: 2018-11-27 11:00

Using cast() or from_address() to convert from c_void_p (or integer) address to POINTER(c_ubyte) causes the interpreter to crash, when accessing the contents of the array.

The problem seems to happen when running the following combination:
Windows 10 @ 64 bit, version 1803 (OS build 17134.407) and python 3 64 bit, starting with 3.5 until 3.7 (the last that was tested).

The following code works fine on the same system using python 2.7.15 or python 3 until 3.4 (inclusive), 64 bit.
In addition, it works well also under Windows 7 64 bit and python 3.5/3.6/3.7 etc.

How to reproduce?
from ctypes import wintypes, windll, c_void_p, c_size_t, POINTER, c_ubyte, cast
# Define constants.
# Configure function arguments.
windll.kernel32.CreateFileMappingA.argtypes = [wintypes.HANDLE, c_void_p, wintypes.DWORD, wintypes.DWORD, wintypes.DWORD, wintypes.LPCSTR]
windll.kernel32.CreateFileMappingA.restype = wintypes.HANDLE
windll.kernel32.MapViewOfFile.argtypes = [wintypes.HANDLE, wintypes.DWORD, wintypes.DWORD, wintypes.DWORD, c_size_t]
windll.kernel32.MapViewOfFile.restypes = wintypes.LPVOID
# Open shared-memory.
handle = windll.kernel32.CreateFileMappingA(-1, None, PAGE_READWRITE, 0, 1024 * 1024, b'TestSHMEM')
# Obtain pointer to SHMEM buffer.
ptr = windll.kernel32.MapViewOfFile(handle, FILE_MAP_ALL_ACCESS, 0, 0, 1024 * 1024)
arr = cast(ptr, POINTER(c_ubyte))
# or
#arr = POINTER(c_ubyte).from_address(ptr)
# now crashes

Also note that changing the return type of MapViewOfFile as follows:
windll.kernel32.MapViewOfFile.restypes = wintypes.LPVOID
is changed to:
windll.kernel32.MapViewOfFile.restypes = POINTER(c_ubyte)
The contents of the buffer can be directly accessed without a problem.

Just in case I'm not using it properly with new python releases?

msg330539 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2018-11-27 17:11
The default function result type is c_int, which truncates a 64-bit pointer to 32-bit. The attribute that needs to be set is singular restype, not plural restypes. Unfortunately ctypes objects have a dict, so you're not catching the typo in an obvious way.

Additional comments:

POINTER(c_ubyte).from_address(ptr) is not the same as cast(ptr, POINTER(c_ubyte)). The first one wrongly instantiates a pointer from the value at the ptr address instead of the address itself. If you want to use from_address, where ptr is a Python integer, a correct expression for this is POINTER(c_ubyte)(c_ubyte.from_address(ptr)).

I recommend using kernel32 = WinDLL('kernel32', use_last_error=True) instead of windll. This allows using ctypes.get_last_error() instead of kernel32.GetLastError(), which is especially more reliable when working in the REPL. It also avoids the problem with cached libraries on the global windll object, which can lead to prototype conflicts between libraries (e.g. if a script makes use of your library and another that uses the Windows API via ctypes, such as colorama), since libraries in turn cache function pointers.
msg330618 - (view) Author: Mordechai Botrashvily (Mordechai Botrashvily) Date: 2018-11-28 17:26
Eryk, thanks :).
This really solved the problem.
Odd how it went unnoticed.
Date User Action Args
2018-11-28 17:26:00Mordechai Botrashvilysetmessages: + msg330618
2018-11-27 17:11:07eryksunsetstatus: open -> closed

nosy: + eryksun
messages: + msg330539

resolution: not a bug
stage: resolved
2018-11-27 11:00:03Mordechai Botrashvilycreate