Title: 32 bit ctypes stdcall callback fails to restore stack pointer
Type: crash Stage: test needed
Components: ctypes, Windows Versions: Python 3.9, Python 3.8
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: David Heffernan, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2019-11-08 16:57 by David Heffernan, last changed 2019-11-22 23:49 by steve.dower.

Messages (2)
msg356249 - (view) Author: David Heffernan (David Heffernan) Date: 2019-11-08 16:57
Starting with Python 3.8 certain ctypes callbacks fail to restore the stack pointer.

In the repo below, when the DLL is compiled with MSVC under default debug settings, running the Python script leads to a debug error dialog which says:

Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call.  This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

It appears that when the C code calls the callback function, the value of ESP is 4 greater than it should be.

This problem does not occur with older versions of Python.

**DLL code**

#include <Windows.h>

BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)
    switch (ul_reason_for_call)
    return TRUE;

typedef void (__stdcall *MYCALLBACK)(int, double);

extern "C"
    __declspec(dllexport) void __stdcall foo(MYCALLBACK callback)
        callback(1, 11);
        callback(2, 21);
        callback(3, 31);

**Python code**

import ctypes
import ctypes.wintypes

def CallbackType(restype, *argtypes):

    def from_param(cls, obj):
        if obj is None:
            return obj
        return ctypes._CFuncPtr.from_param(obj)

    result = ctypes.WINFUNCTYPE(restype, *argtypes)
    result.from_param = classmethod(from_param)
    return result

MYCALLBACK = CallbackType(

def callback(handle, time):
    print(handle, time)
mycallback = MYCALLBACK(callback)

lib = ctypes.WinDLL(r'path\to\dll\foo.dll')
func = getattr(lib, '_foo@4')
func.restype = None
func.argtypes = MYCALLBACK,
msg357345 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-11-22 23:49
I would guess this is due to the updated libffi, and probably it's a case that is not sufficiently tested.

Adding tests is the first step. It'll probably have to enumerate many parameter types (in case there's something weird here like misaligning the double after the int and not clearing it up, as opposed to a constant 4-byte offset).

Hopefully then the issue is on our side and not part of libffi, but it could be anywhere.
Date User Action Args
2019-11-22 23:49:58steve.dowersetstage: test needed
messages: + msg357345
versions: + Python 3.9
2019-11-22 15:08:47eryksunsetnosy: + paul.moore, tim.golden, zach.ware, steve.dower
components: + Windows
2019-11-08 16:57:34David Heffernancreate