With Python 2.7.8, ctypes builds fine on FreeBSD x86, but with Python 2.7.9, the build fails with:

*** WARNING: renaming "_ctypes" since importing it failed: build/lib.freebsd-8.3-RELEASE-p3-i386-2.7/_ctypes.so: Undefined symbol "ffi_call_win32"

Since this is FreeBSD, there shouldn't really be any calls to ffi_call_win32() in the _ctypes module or libffi.

Looking at the code changes in libffi/src/x86/ffi.c, it looks as if some of the #ifdefs for X86_WIN64 and X86_WIN32 were mixed up, causing calls to the Windows function to be compiled on FreeBSD x86, without also compiling the function definition in the same file.

The cause seems to be these changes of the file (diff between the 2.7.8 and 2.7.9 version):

@@ -368,14 +374,21 @@ void ffi_call(ffi_cif *cif, void (*fn)(v
 #ifdef X86_WIN64
     case FFI_WIN64:
       ffi_call_win64(ffi_prep_args, &ecif, cif->bytes,
                      cif->flags, ecif.rvalue, fn);
       break;
-#elif defined(X86_WIN32)
+#else
+#ifndef X86_WIN32
+    case FFI_SYSV:
+      ffi_call_SYSV(ffi_prep_args, &ecif, cif->bytes, cif->flags, ecif.rvalue,
+                    fn);
+      break;
+#else
     case FFI_SYSV:
-    case FFI_STDCALL:
     case FFI_MS_CDECL:
+#endif
+    case FFI_STDCALL:
       ffi_call_win32(ffi_prep_args, &ecif, cif->abi, cif->bytes, cif->flags,
                     ecif.rvalue, fn);
       break;
     case FFI_THISCALL:
     case FFI_FASTCALL:
@@ -404,15 +417,10 @@ void ffi_call(ffi_cif *cif, void (*fn)(v
          abi = FFI_STDCALL;
         ffi_call_win32(ffi_prep_args, &ecif, abi, cif->bytes, cif->flags,
                        ecif.rvalue, fn);
       }
       break;
-#else
-    case FFI_SYSV:
-      ffi_call_SYSV(ffi_prep_args, &ecif, cif->bytes, cif->flags, ecif.rvalue,
-                    fn);
-      break;
 #endif
     default:
       FFI_ASSERT(0);
       break;
     }
@@ -785,14 +797,21 @@ ffi_raw_call(ffi_cif *cif, void (*fn)(vo
     ecif.rvalue = rvalue;


   switch (cif->abi)
     {
-#ifdef X86_WIN32
+#ifndef X86_WIN32
+    case FFI_SYSV:
+      ffi_call_SYSV(ffi_prep_args_raw, &ecif, cif->bytes, cif->flags,
+                    ecif.rvalue, fn);
+      break;
+#else
     case FFI_SYSV:
-    case FFI_STDCALL:
     case FFI_MS_CDECL:
+#endif
+#ifndef X86_WIN64
+    case FFI_STDCALL:
       ffi_call_win32(ffi_prep_args_raw, &ecif, cif->abi, cif->bytes, cif->flags,
                     ecif.rvalue, fn);
       break;
     case FFI_THISCALL:
     case FFI_FASTCALL:
@@ -821,15 +840,10 @@ ffi_raw_call(ffi_cif *cif, void (*fn)(vo
          cif->abi = abi = FFI_STDCALL;
         ffi_call_win32(ffi_prep_args_raw, &ecif, abi, cif->bytes, cif->flags,
                        ecif.rvalue, fn);
       }
       break;
-#else
-    case FFI_SYSV:
-      ffi_call_SYSV(ffi_prep_args_raw, &ecif, cif->bytes, cif->flags,
-                    ecif.rvalue, fn);
-      break;
 #endif
     default:
       FFI_ASSERT(0);
       break;
     }

If neither X86_WIN64 nor X86_WIN32 are defined, the #ifndefs cause the functions calls to be compiled in.

@lemburg, does reverting this patch fix the regression for you?

Faced the same problem on FreeBSD.

I confirm: applying the reversed patch fixed it.

Thanks!

Python 3.4 and 3.5 are also affected: I marked issue #22521 as a duplicate of this issue.

To provide supporting information, using the i386 release of FreeBSD 10.1:

* ctypes fails to build with the latest from Python 2.7 branch with the message:

    build/temp.freebsd-10.1-RELEASE-i386-2.7-pydebug/usr/home/davin/pycoredev/cpython/Modules/_ctypes/libffi/src/x86/ffi.o: In function `ffi_prep_closure_loc':
    /usr/home/davin/pycoredev/cpython/Modules/_ctypes/libffi/src/x86/ffi.c:677: undefined reference to `ffi_closure_FASTCALL'
    /usr/home/davin/pycoredev/cpython/Modules/_ctypes/libffi/src/x86/ffi.c:683: undefined reference to `ffi_closure_THISCALL'
    /usr/home/davin/pycoredev/cpython/Modules/_ctypes/libffi/src/x86/ffi.c:689: undefined reference to `ffi_closure_STDCALL'
    /usr/bin/ld: build/lib.freebsd-10.1-RELEASE-i386-2.7-pydebug/_ctypes.so: hidden symbol `ffi_closure_FASTCALL' isn't defined
    /usr/bin/ld: final link failed: Nonrepresentable section on output
    cc: error: linker command failed with exit code 1 (use -v to see invocation)
    
    ...
    
    Failed to build these modules:
    _ctypes                                               


* Note that the above failure message was described in issue22521 by @Danya.Alexeyevsky

* ctypes also fails to build with the latest from default/3.5 branch with the very same message


Apparently the compilation failure messages differ between FreeBSD 8.3 and 10.1 (both i386).

Reading issue22634 suggests that perhaps this issue is not limited to FreeBSD -- issue22634 appears to suffer the same behavior on RHEL 6.4 and 5.5 when compiling to 32-bit (the -m32 compiler flag is being used there).

Attaching a patch which enabled successful compilation of ctypes on FreeBSD 10.1 i386-RELEASE in each of default/3.5, 3.4, and 2.7 branches.

This patch attempts to fix the incomplete logic regarding Windows target platforms in the ifdef's inside Modules/_ctypes/libffi/src/x86/ffi.c which at times tested for X86_WIN64 but not X86_WIN32 inside corresponding #else clauses (and vice versa).

This especially needs to be tested on a 32-bit Windows system -- I do not have one handy so help testing it there (and elsewhere) would be much appreciated.

Inheriting the priority from issue22634 which has been marked closed->duplicate (duplicate of this issue).

Previously issue22634 was given a priority of "release blocker" by @pitrou.  Issue22634 described how this issue also afflicts 32-bit builds on multiple RHEL release versions.

Modules/_ctypes/libffi in branches 2.7, 3.4, default is currently a copy of libffi 3.1.
Is problem reproducible with libffi 3.2.1? If not, then it might be better to update Modules/_ctypes/libffi to libffi 3.2.1.

I would prefer to have someone with good libffi knowledge to chime in here. It's easy to just revert the patch, but it may be even better to upgrade the included libffi lib.

But probably not to 3.2.1, since this still has the same bug it seems:

https://github.com/atgreen/libffi/blob/v3.2.1/src/x86/ffi.c#L401

@Marc, if upstream 3.2.1 also has the issue, then that would mean the current FreeBSD Python ports, which use --use-system-ffi and the security/libffi port, currently at version 3.2.1 [1], can reproduce the issue. 

I'm not aware of reports that they fail in this manner though.

[1] http://www.freshports.org/devel/libffi/

See also: 

Issue: https://github.com/atgreen/libffi/issues/180
Title: OSX/i386 build is broken in v3.2.1

@koobs: I was just looking at the 3.2.1 code where it still looks like ffi_call_win32() gets called even on non-Win32 platforms.

That said, it's possible that ffi_call_win32() is indeed defined on other platforms than Win32 as well in 3.2.1 - after all, it just implements a particular calling method. I haven't checked that or actually tried compiling _ctypes with 3.2.1.

Perhaps we could try switch to 3.2.1 and then check the buildbots for possible issues ?!

@marc I took a look at the code upstream and it does indeed appear to be the same. It was introduced in 3.1 [1].

I cant explain however how or why our Python ports work with libffi 3.2.1.

See msg238767 for a link to another similar (same?) issue, with failure of OSX on 3.2.1 (building libffi, not python)

[1] https://github.com/atgreen/libffi/commit/e1911f78df113ca58738b66089a070d4cf747de7

Here's a patch which I have tested on Linux, FreeBSD and Mac OS X.

It solves the problem with compiling in Windows calls on non-Windows platforms and resynchronizes the ffi_raw_call() function with the ffi_call() implementation. Both functions had the same issue and the only difference between the two is the use of ffi_prep_args_raw instead of ffi_prep_args.

As with davin's patch, this would need to be tested on Windows.

Let's see what the buildbots think.

New changeset 96a7b401d5e4 by Benjamin Peterson in branch '2.7':
fix libffi compilation on FreeBSD (#23042)
https://hg.python.org/cpython/rev/96a7b401d5e4

New changeset a04b3de18c4c by Benjamin Peterson in branch '3.4':
fix libffi compilation on FreeBSD (#23042)
https://hg.python.org/cpython/rev/a04b3de18c4c

New changeset 987b30a88653 by Benjamin Peterson in branch 'default':
merge 3.4 (#23042)
https://hg.python.org/cpython/rev/987b30a88653