issue for tracking the libffi-3.0.12 import. checked that builds on x86_64-linux-gnu and arm-linux-gnueabihf do work.

still needs updating/checking the extra copies of:

 - libffi_arm_wince
 - libffi_msvc
 - libffi_osx

New changeset 7727be7613f9 by doko in branch 'default':
- Issue #17192: Import libffi-3.0.12.
http://hg.python.org/cpython/rev/7727be7613f9

libffi_osx is not a copy of the regular libffi, but a (fairly old) fork. I don't know how far the two "branches" have diverged.  

An important feature  of liffi_osx is that is compiles cleanly when all intel and ppc related sources are compiled with '-arch ppc -arch i386 -arch x86_64' (that is, the relevant sources contain preprocessor guards to ensure the code in those files is only compiled for the right archictures). This greatly simplifies the build process because it is not necessary to build libffi a couple of times and then merge the resulting binaries.

I'm -0 at this point w.r.t. trying to remove libffi_osx, that tree works and I'll work on it anyway (it is also used in PyObjC).

Maybe Modules/_ctypes/libffi in 2.7, 3.2 and 3.3 branches should be updated?

Sound like a large change for stable releases.  Usually we would not do that.

However internal copy of expat was updated in 2.7 and 3.2 branches about 2 weeks ago (issue #14340).

An update to libffi is needed for all maintained versions of Python.

In 2.7, we're running into the stack being misaligned in 32-bit x86 code which is something a libffi update fixes.  It is a simple patch:

 http://patchwork.ozlabs.org/patch/58128/

which comes to the official libffi releases via https://github.com/atgreen/libffi/commit/3f5b1375ab1e2b8e3d593e21b27097a4a50f9b83#src/x86/sysv.S.

The problem: without the stack being 16-byte aligned, code generated by modern compilers like recent gcc/g++ or clang assumed that the stack is 16 byte aligned and uses SSE instructions in some circumstances that require this.  Without this fix, any ctypes call into such code will crash.

Sure, that tiny patch could be cherry picked into our libffi, but we IMNSHO may as well just update the entire thing given that we embed a very old copy rather than use it as an external dependency.

http://bugs.python.org/issue17245 filed to track the stack alignment issue.  The only reason i set this as release blocker is to let a release manager decide which of these two issues to proceed with for 2.7.4 and 3.2.4 (and 3.3.1).

By all means, upgrade it.

before the updates, ... there seem to be two test failures on sparc solaris. the local libffi/src/sparc/v8.S change was integrated upstream, so I don't yet what could cause these failures. or did they fail before too?

Updating 3.x branches is fine with me.

New changeset b10ec5083a53 by Gregory P. Smith in branch '2.7':
Fixes issue #17192: Update the ctypes module's libffi to v3.0.13.  This
http://hg.python.org/cpython/rev/b10ec5083a53

New changeset 43e8da639462 by Gregory P. Smith in branch '3.2':
Fixes issue #17192: Update the ctypes module's libffi to v3.0.13.  This
http://hg.python.org/cpython/rev/43e8da639462

New changeset f23274357fc8 by Gregory P. Smith in branch '3.3':
Fixes Issue #17192: Update the ctypes module's libffi to v3.0.13.  This
http://hg.python.org/cpython/rev/f23274357fc8

New changeset a94b3b4599f1 by Gregory P. Smith in branch 'default':
Fixes Issue #17192: Update the ctypes module's libffi to v3.0.13.  This
http://hg.python.org/cpython/rev/a94b3b4599f1

2.7, 3.2, 3.3 and 3.4 have all be updated to ctypes v3.0.13.

fwiw, I tried a build of 3.3 on Sparc Solaris 10 using snakebite (nitrogen) and ctypes builds and links but two tests fail: ctypes.test.test_bitfields.C_Test test_ints and test_shorts failed.

That should be dealt with via a separate issue and by someone who cares about Solaris.

These break what was addressed in #11729 for default, 3,x and 3.3.

2.7 seems to have made it through unscathed.

I'm not sure where or how the old code was introduced, but the clang fix has been upstreamed and is correct in the pure libffi 3.0.13 sources

Failure to build ctypes can be seen here:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.x/builds/1246/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.x/builds/1245/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.3/builds/538/steps/test/logs/stdio

Greg?

it _looks_ like the libffi updates i put in 3.2, 3.3 and default (3.4) are messed up with some changes missing in 3.2 and more missing in 3.3 and 3.4.  (i'm comparing vs upstream 3.0.13 <-> 2.7 <-> 3.2 <-> 3.3 <-> 3.4).

i blame the mess of merging that was required by hg due to the plethora of versions of libffi we had in different states in 2.7, 3.2 and 3.3+ before i did the update.  i'm fixing it now.

Thanks. For 3.2, please attach a patch here.

I've got this and all of the fixes necessary for 3.3, default and 2.7 in my local repositories.

odd thing in the 3.2 patch: it is entirely unclear to me if the issue 10309 fix needs to be kept.  i included it as it was missing from 3.2 after previous commits (3.3 and 3.4 still kept it).  2.7 needs it as well.  the other part of the patch is the issue 11729 fix that already exists upstream being reapplied; it was inadvertently undone during the upgrade.

I _could_ push fixes for all branches at once right now unless pushes to 3.2 are blocked and prevent me from pushing anything.

(rant: at which point i declare "hg hell" and feel like giving up because it simply has gotten in the way throughout this entire process rather than helping... i'll be forced to manually create diffs and reapply them to _other_ hg checkouts?  not so useful.  surely someone will stupidly claim i'm holding hg wrong instead of the tool being unhelpful).

We need to GET RID of libffi being bundled as part of the source release in 3.4.  this is disgusting.

OK, 3.2 block disabled, push away.

Ready to confirm on buildbots when this lands

Also, libffi-3.0.13 is installed on my buildbots if you want to use -custom to have a play, I can be pinged anytime on IRC if I can be of any assistance. koobs @ #python-dev

New changeset 7a2e0c1bb1a9 by Gregory P. Smith in branch '3.2':
* Fix issue 17192 for 3.2 - reapply the issue11729 patch that was undone
http://hg.python.org/cpython/rev/7a2e0c1bb1a9

New changeset aa3371fa9773 by Gregory P. Smith in branch '3.3':
* Fix PART of issue 17192 for 3.3 - reapply the issue11729 patch
http://hg.python.org/cpython/rev/aa3371fa9773

New changeset e0fdc21c7d13 by Gregory P. Smith in branch 'default':
* Fix PART of issue 17192 for 3.4 - reapply the issue11729 patch
http://hg.python.org/cpython/rev/e0fdc21c7d13

New changeset c665abe15c4a by Gregory P. Smith in branch '3.3':
* Fix remaining bits of issue 17192 for 3.3 - these changes
http://hg.python.org/cpython/rev/c665abe15c4a

New changeset 688bc0b44d96 by Gregory P. Smith in branch 'default':
* Fix remaining bits of issue 17192 for 3.4 - these changes
http://hg.python.org/cpython/rev/688bc0b44d96

New changeset 663800e18fef by Gregory P. Smith in branch '3.2':
This local change was lost during the fixing of issue17192 to update
http://hg.python.org/cpython/rev/663800e18fef

New changeset 7438d202e380 by Gregory P. Smith in branch '2.7':
This local change was lost during the fixing of issue17192 to update
http://hg.python.org/cpython/rev/7438d202e380