classification
Title: callback function on win64 results in bad behavior. mem corruption?
Type: crash Stage: committed/rejected
Components: ctypes Versions: Python 3.2, Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: cgohlke, georg.brandl, ocrush, pitrou, srid, stan.mihai, theller
Priority: normal Keywords: patch

Created on 2010-03-31 18:53 by ocrush, last changed 2011-01-31 21:48 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
ctype_win64.txt ocrush, 2010-03-31 18:53 Contains Python and C code. comment the print statement in jfunc to fix the issue.
issue8275_win64_ctypes_no_optimization.patch srid, 2010-12-22 02:01
ctypes_win64.diff stan.mihai, 2011-01-26 09:14 one-liner fix
ctypes_win64_tests.diff stan.mihai, 2011-01-31 15:43 patch with tests
Messages (11)
msg102028 - (view) Author: Jasmit (ocrush) Date: 2010-03-31 18:53
I am testing a wrapper on Windows 64 and it seems to result in an null pointer access error ONLY when I insert a print statement in the C code.  I have tested the wrapper with Python 2.6 and Python 2.7a4.  In addition, I have compiled Python 2.6.5 source code and ONLY the release version results in an error.  I think the issue is with memcpy(obj->b_ptr, *pArgs, dict->size) (callbacks.c).  pArgs seem to be  corrupted.  However, I am only looking at the code for the first time and I might be off base.

The following is Python and C code to reproduce the bug.  To resolve, please comment printf statement in jfunc (C function).

Python Code:
from ctypes import *
def fcn(m,n,x,f):
    print "IN Python function fcn ................"
    print f[0]
    
m=3
n=1
pydlltest=cdll.pydlltest
pydlltest.jfunc.restype = POINTER(c_double)
evalstring = 'pydlltest.jfunc('
TMP_FCN=CFUNCTYPE(None,c_int,c_int,POINTER(c_double),POINTER(c_double))
tmp_fcn=TMP_FCN(fcn)
state=[TMP_FCN,tmp_fcn]
evalstring += 'tmp_fcn'
evalstring +=','
evalstring +='c_int(m)'
evalstring +=','
evalstring +='c_int(n)'
evalstring += ')'
print "evalstring=",evalstring
result = eval(evalstring)

C code:
#include <stdio.h>
__declspec(dllexport) double *jfunc(void (*fcn) (int,int,double [],double[]),int m,int n);

double *jfunc(void (*fcn) (int,int,double [],double []),int m,int n)
{
	double *fvec=NULL;
	double *xguess = NULL;
	int i = 0;
	int j = 0;
	/* comment the line below to fix the resulting null pointer access error */
	printf("In j func .................\n");
	fvec = (double *) malloc (m * sizeof (double));
	xguess = (double *) malloc (n * sizeof (double));
	for (i = 0; i < n; i++){
	   xguess[i] = 0.123;
	}
	(*fcn) (m, n, xguess, fvec);
	return fvec;
}
msg102544 - (view) Author: Jasmit (ocrush) Date: 2010-04-07 15:59
I was able to fix the issue by modifying and re-compiling ctypes project in Visual Studio 2008.  The following properties were modified:

Release:
configuration Properties->C/C++->Optimization->Optimization: Disabled(/Od)
configuration Properties->C/C++->Optimization->Enable Intrinsic Functions:  No
msg124474 - (view) Author: Sridhar Ratnakumar (srid) Date: 2010-12-22 02:01
Attaching a patch for the configuration changes mentioned in msg102544
msg126713 - (view) Author: Christoph Gohlke (cgohlke) Date: 2011-01-21 10:04
This patch fixes issue #9884 and possibly #9266.
msg127092 - (view) Author: stan mihai (stan.mihai) Date: 2011-01-26 09:14
Disabling optimizations doesn't really fix the issue, just hides it, for now.

The problem was an uninitialized variable. Attached is the patch that fixes it.
msg127121 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-26 15:10
Would it be possible to add a test case to the patch?
msg127127 - (view) Author: Christoph Gohlke (cgohlke) Date: 2011-01-26 18:12
Thank you. The new patch works and it also fixes a crash of the 	python-2.5.4.amd64 interpreter at startup when ctypes 1.0.2 and pyreadline 1.6.2 are installed.
msg127605 - (view) Author: stan mihai (stan.mihai) Date: 2011-01-31 15:43
attached patch with tests

I have no experience with python development so the tests will actually need a more detailed review. Also I only checked it on win64.
msg127610 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-31 16:16
Patch verified to work under Linux and win32. It looks good, except for tab characters in the Modules/_ctypes/_ctypes_test.c (indentation of C files should use 4 spaces); that's a detail that we can fix ourselves.
msg127657 - (view) Author: stan mihai (stan.mihai) Date: 2011-01-31 20:35
ok, please also fix the first parameter of the tests. Because it has the same value in the first call and the callback it will always be right by accident, since the first call puts the value in both standard and floating point registers.
msg127668 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-31 21:48
Ok, committed in r88284 (3.2), r88285 (3.1) and r88286 (2.7). Thank you!
History
Date User Action Args
2011-01-31 21:48:33pitrousetstatus: open -> closed
nosy: theller, georg.brandl, pitrou, srid, cgohlke, ocrush, stan.mihai
messages: + msg127668

resolution: fixed
stage: patch review -> committed/rejected
2011-01-31 20:35:31stan.mihaisetnosy: theller, georg.brandl, pitrou, srid, cgohlke, ocrush, stan.mihai
messages: + msg127657
2011-01-31 16:16:47pitrousetassignee: theller ->
messages: + msg127610
nosy: theller, georg.brandl, pitrou, srid, cgohlke, ocrush, stan.mihai
2011-01-31 15:43:27stan.mihaisetfiles: + ctypes_win64_tests.diff
nosy: theller, georg.brandl, pitrou, srid, cgohlke, ocrush, stan.mihai
messages: + msg127605
2011-01-26 18:12:39cgohlkesetnosy: theller, georg.brandl, pitrou, srid, cgohlke, ocrush, stan.mihai
messages: + msg127127
2011-01-26 15:10:23pitrousetversions: + Python 3.1, Python 3.2, - Python 2.6
nosy: + pitrou, georg.brandl

messages: + msg127121

type: behavior -> crash
stage: patch review
2011-01-26 09:14:40stan.mihaisetfiles: + ctypes_win64.diff
nosy: + stan.mihai
messages: + msg127092

2011-01-21 10:04:16cgohlkesetnosy: theller, srid, cgohlke, ocrush
messages: + msg126713
2011-01-21 03:07:57cgohlkesetnosy: + cgohlke
2010-12-22 02:01:19sridsetfiles: + issue8275_win64_ctypes_no_optimization.patch

nosy: + srid
messages: + msg124474

keywords: + patch
2010-04-07 15:59:28ocrushsetmessages: + msg102544
2010-03-31 20:18:20ocrushsettitle: callback function on win64 results in bad behavior. mem leak? -> callback function on win64 results in bad behavior. mem corruption?
2010-03-31 18:53:44ocrushcreate