test_ctypes fails when building and testing on OSX 10.7 (Lion).

This is likely caused by changes in the compiler w.r.t. the calling convention. I've worked around this problem in PyObjC, and will try to port that workaround to ctypes.

BTW. Using --with-system-ffi does not fix the issue for me

test_ctypes fails when building and testing on OSX 10.7 (Lion).

What is the error message?

I should have included the failures in my first message, sorry about that.

IIRC the workaround in PyObjC I mention in my initial message does explicit sign extension when needed.

W.r.t to the first couple of failures, not that:

struct.unpack('b', struct.pack('B', 130)) == (-126,)

======================================================================
FAIL: test_byte (ctypes.test.test_cfuncs.CFunctions)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_cfuncs.py", line 21, in test_byte
    self.assertEqual(self.S(), -126)
AssertionError: 130 != -126

======================================================================
FAIL: test_byte_plus (ctypes.test.test_cfuncs.CFunctions)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_cfuncs.py", line 27, in test_byte_plus
    self.assertEqual(self.S(), -126)
AssertionError: 130 != -126

======================================================================
FAIL: test_short (ctypes.test.test_cfuncs.CFunctions)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_cfuncs.py", line 45, in test_short
    self.assertEqual(self.S(), -32766)
AssertionError: 32770 != -32766

======================================================================
FAIL: test_short_plus (ctypes.test.test_cfuncs.CFunctions)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_cfuncs.py", line 51, in test_short_plus
    self.assertEqual(self.S(), -32766)
AssertionError: 32770 != -32766

======================================================================
FAIL: test_doubleresult (ctypes.test.test_functions.FunctionTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_functions.py", line 143, in test_doubleresult
    self.assertEqual(result, -21)
AssertionError: 65771.0 != -21

======================================================================
FAIL: test_floatresult (ctypes.test.test_functions.FunctionTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_functions.py", line 131, in test_floatresult
    self.assertEqual(result, -21)
AssertionError: 65771.0 != -21

======================================================================
FAIL: test_intresult (ctypes.test.test_functions.FunctionTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_functions.py", line 105, in test_intresult
    self.assertEqual(result, -21)
AssertionError: 65771 != -21

======================================================================
FAIL: test_longdoubleresult (ctypes.test.test_functions.FunctionTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/2.7/lib/python2.7/ctypes/test/test_functions.py", line 155, in test_longdoubleresult
    self.assertEqual(result, -21)
AssertionError: 65771.0 != -21

I can't reproduce this on Lion using the 2.7 or default branches. I can't build with GCC at the moment because of bpo-13241. I am using the clang that ships with Lion:

motherbrain:cpython meadori$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.7.2
BuildVersion: 11C74
motherbrain:cpython meadori$ clang --version
Apple clang version 3.0 (tags/Apple/clang-211.12) (based on LLVM 3.0svn)
Target: x86_64-apple-darwin11.2.0
Thread model: posix

It looks like the test failures do not show up with debug enabled (--with-pydebug). I'm able to reproduce them with:

./configure MACOSX_DEPLOYMENT_TARGET=10.7 CC=clang

I can reproduce it without --with-pydebug. Thanks. I will investigate.

The reason for this issue appears to be a bug in the clang optimizer. This is why it only shows up with debug disabled. When I compile with -O0 instead of -O3, the test passes.

This issue from the clang tracker may be the same issue:
http://llvm.org/bugs/show_bug.cgi?id=9524

So I have debugged the first failure and it has to do with the sign extending of 'signed char' parameters. The first test case basically does this:

   _dll = CDLL(_ctypes_test.__file__)
   _dll.tf_b.restype = c_byte
   _dll.tf_b.argtypes = (c_byte,)
   _dll.tf_b(-126)

where 'tf_b' is defined in C code like:

signed char tf_b(signed char c) { return c/3; }

Clang with -O3 generates code for 'tf_b' like:

0x0000000100000ed0 <tf_b+0>: push %rbp
0x0000000100000ed1 <tf_b+1>: mov %rsp,%rbp
0x0000000100000ed4 <tf_b+4>: movslq %edi,%rax
0x0000000100000ed7 <tf_b+7>: imul $0x55555556,%rax,%rax
0x0000000100000ede <tf_b+14>: mov %rax,%rcx
0x0000000100000ee1 <tf_b+17>: shr $0x3f,%rcx
0x0000000100000ee5 <tf_b+21>: shr $0x20,%rax
0x0000000100000ee9 <tf_b+25>: add %ecx,%eax
0x0000000100000eeb <tf_b+27>: movsbl %al,%eax
0x0000000100000eee <tf_b+30>: pop %rbp
0x0000000100000eef <tf_b+31>: retq

See how 'movslq' is used to sign extend the first argument? Since the first argument is a 'signed char', that should be 'movsbq'. I am pretty sure this is a clang bug.

I am going to see if this is fixed in later versions of clang. I will also search the clang tracker to see if this has been reported.

Since the first argument is a 'signed char', that should be 'movsbq'.

Hmmm, on second thought I am not 100% sure about that exact statement, but I still think this is most likely a clang bug having something to do with sign extension.

It turns out that this isn't OS X specific. I can reproduce the exact same issue by building with clang on a 64-bit Fedora 16 box.

Thanks for the analysis, Meador. So perhaps as a temporary workaround, we could tweak setup.py to set -O0 for building ctypes with clang?

So perhaps as a temporary workaround, we could tweak setup.py to set
-O0 for building ctypes with clang?

That would make the tests pass, but it still won't fix the fundamental
issue. clang and GCC have a difference in opinion as to when function
parameters and return values should be sign extended. libffi follows
the GCC model.

I am talking to the LLVM folks right now to see why things are
implemented this way. Whether LLVM or libffi should be fixed depends
on the outcome of that discussion.

Due to this ABI difference compiling _ctypes with -O0 for clang will not
fix the problem because any ctypes code that interacts with C code that
was compiled by clang with optimizations has the potential to break just
like our test cases.

LLVM/clang bug report here: http://llvm.org/bugs/show_bug.cgi?id=12207.

An update: test_ctypes continues to fail on 2.7.3rc2 with exactly the same failures as originally reported when compiled non-debug 64-bit (x86_64) with either the latest clang *or* llvm-gcc from Xcode 4.3.1. It does not fail when compiled with gcc-4.2 from Xcode 3.2.6. Not does it fail with any of the three compilers when compiled and run as 32-bit (i386). So it does seem to come down to a difference between standard gcc and with the llvm-based compilers (clang, llvm-gcc), and not just clang.

Python 2.7.3rc2 (2.7:f59ab537a5dc, Mar 17 2012, 15:26:37)
[GCC 4.2.1 Compatible Apple Clang 3.1 (tags/Apple/clang-318.0.54)] on darwin

Python 2.7.3rc2 (2.7:f59ab537a5dc, Mar 17 2012, 15:47:25)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.9.00)] on darwin

Python 2.7.3rc2 (2.7:f59ab537a5dc, Mar 17 2012, 16:08:58)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

FYI: This bug still exists using the version of clang shipped with OS X 10.8.

Tested with Python 3.3.0b2 and:
Apple clang version 4.0 (tags/Apple/clang-421.0.57) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin12.0.0
Thread model: posix

The attached patch syncs libffi_osx with the copy of libffi that's included with PyObjC.

With this patch test_ctypes passes for me when using clang to build on OSX 10.8.

I haven't run the tests with other python releases yet, and also not on a OSX 10.5 system (where I can test if the patch works with GCC as well)

I've tested the patch on OSX 10.5 with GCC and all tests pass.

I'd like to get this into 3.3rc1 as the test failure indicate a problem that could cause very vague problems for users of the ctypes library that use clang on OSX to build python (which includes anyone building python from source on OSX 10.7 or later)

I can also confirm that the patch applied to the current tip (663de4dbb88a) passes test_ctypes now with Clang 4.0 on OS X 10.8.

The patch as it stands causes _ctypes build failures when using the 10.4u SDK as in the traditional 10.3+ 32-bit-only installer configuration. Apparently, in ppc-ffi_darwin.c, the skip that the patch removes was because OSCacheControl.h doesn't exist in the 10.4u SDK. It is in the 10.5 SDK and _ctypes.so builds and tests OK on both PPC and i386 with it.

--- a/Modules/_ctypes/libffi_osx/powerpc/ppc-ffi_darwin.c
+++ b/Modules/_ctypes/libffi_osx/powerpc/ppc-ffi_darwin.c
@@ -37,19 +37,10 @@
 #include <ppc-darwin.h>
 #include <architecture/ppc/mode_independent_asm.h>
 
-#if 0
 #if defined(POWERPC_DARWIN)
 #include <libkern/OSCacheControl.h>	// for sys_icache_invalidate()
 #endif
 
-#else
-
-#pragma weak sys_icache_invalidate
-extern void sys_icache_invalidate(void *start, size_t len);
-
-#endif

./Modules/_ctypes/libffi_osx/powerpc/ppc-ffi_darwin.c:41:67: error: libkern/OSCacheControl.h: No such file or directory

The change to ppc-ffi_darwin.c was unintentional and should be removed. I must have been sleeping during the merge, as I only intended to copy the files for i386/x86_64 and not for PPC as the latter architecture didn't have problems when compiled with clang.

I've copied the ppc-ffi_darwin.c file from Python's tree into the PyObjC tree to avoid problems when there are future merges.

I just noticed another anomaly in the patch. In x86-ffi_darwin.c there appears to be some bogus code duplicating ffi_prep_args. Could you review and clean up the patch before we apply it?

I didn't have time to look into this today and will look into this tomorrow.

What code duplication do you mean?

The code in the switch statement at line 71 looks similar for the various case, but it isn't: there is variation in the types uses in the casts (and particularly in the cast used to read the input value).

Attached version of the patch without the change to the powerpc specific code.

I will commit this version later today (after another testing round).

Sorry, I meant this part of the patch:

diff --git a/Modules/_ctypes/libffi_osx/x86/x86-ffi_darwin.c b/Modules/_ctypes/libffi_osx/
--- a/Modules/_ctypes/libffi_osx/x86/x86-ffi_darwin.c
+++ b/Modules/_ctypes/libffi_osx/x86/x86-ffi_darwin.c
@@ -35,6 +35,8 @@
 /* ffi_prep_args is called by the assembly routine once stack space
  has been allocated for the function's arguments */
 
+void ffi_prep_args(char *stack, extended_cif *ecif);
+
 void ffi_prep_args(char *stack, extended_cif *ecif)
 {
     register unsigned int i;

That's not code duplication, but a way to silence a compiler warning. I added it while trying to get pyobjc to compile without any warnings.

I noticed earlier today that test_ctypes passes for Python 3.3 both with and without my patch. This is on OSX 10.8.1 with an up-to-date Xcode.

Configure arguments:

'--enable-framework' '--with-pydebug' '--enable-universalsdk=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk' '--with-universal-archs=intel' '--with-framework-name=PythonDev' 'MACOSX_DEPLOYMENT_TARGET=10.8' 'CC=clang' 'CXX=clang++'

That's why I haven't commited the patch yet. I'm installing Xcode in a OSX 10.7 VM to check if the patch is necessary on that platform.

Bah, I just noticed that you built --with-pydebug enabled. That hides the problem. I was fooled myself there. But I've since verified that the failure still exists when built on 10.7 with Xcode 4.4.1 so the patch should go in.

Duh...

3.3 without patch fails when I leave out '--with-pydebug'. I'm now running the testsuite for 2.7 and 3.2 with patch and will commit once those are ready, which should be within an hour from now.

New changeset 58a87c02692c by Ronald Oussoren in branch '2.7':
Fix bpo-13370: Ensure that ctypes works on Mac OS X when Python is compiled using the clang compiler
http://hg.python.org/cpython/rev/58a87c02692c

New changeset a425f2697273 by Ronald Oussoren in branch '3.2':
Fix bpo-13370: Ensure that ctypes works on Mac OS X when Python is compiled using the clang compiler
http://hg.python.org/cpython/rev/a425f2697273

New changeset bed8dd30309d by Ronald Oussoren in branch 'default':
Fix bpo-13370: Ensure that ctypes works on Mac OS X when Python is
http://hg.python.org/cpython/rev/bed8dd30309d

The issue should be gone in the upcoming 3.3rc1 release and the next stable releases of 3.2 and 2.7.