libffi-3.0.13 import #61394

doko42 · 2013-02-12T13:52:53Z

BPO	17192
Nosy	@birkenfeld, @doko42, @gpshead, @ronaldoussoren, @larryhastings, @benjaminp, @merwok, @koobs
Files	issue17192-refix-3.2-gps01.diff: the patch to bring the 3.2 branch libffi back where it should be.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2013-04-30.08:32:24.431>
created_at = <Date 2013-02-12.13:52:52.529>
labels = ['extension-modules', 'type-bug', 'release-blocker']
title = 'libffi-3.0.13 import'
updated_at = <Date 2013-04-30.08:32:24.430>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2013-04-30.08:32:24.430>
actor = 'gregory.p.smith'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2013-04-30.08:32:24.431>
closer = 'gregory.p.smith'
components = ['Extension Modules']
creation = <Date 2013-02-12.13:52:52.529>
creator = 'doko'
dependencies = []
files = ['30076']
hgrepos = []
issue_num = 17192
keywords = ['patch']
message_count = 22.0
messages = ['181956', '181958', '181960', '182047', '182066', '182205', '182440', '182443', '182444', '182448', '182870', '184560', '184693', '184703', '187427', '188017', '188129', '188131', '188135', '188136', '188138', '188139']
nosy_count = 10.0
nosy_names = ['georg.brandl', 'doko', 'gregory.p.smith', 'ronaldoussoren', 'larry', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'python-dev', 'koobs']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17192'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

doko42 · 2013-02-12T13:52:52Z

issue for tracking the libffi-3.0.12 import. checked that builds on x86_64-linux-gnu and arm-linux-gnueabihf do work.

still needs updating/checking the extra copies of:

libffi_arm_wince
libffi_msvc
libffi_osx

python-dev · 2013-02-12T14:33:33Z

New changeset 7727be7613f9 by doko in branch 'default':

Issue bpo-17192: Import libffi-3.0.12.
http://hg.python.org/cpython/rev/7727be7613f9

ronaldoussoren · 2013-02-12T14:43:30Z

libffi_osx is not a copy of the regular libffi, but a (fairly old) fork. I don't know how far the two "branches" have diverged.

An important feature of liffi_osx is that is compiles cleanly when all intel and ppc related sources are compiled with '-arch ppc -arch i386 -arch x86_64' (that is, the relevant sources contain preprocessor guards to ensure the code in those files is only compiled for the right archictures). This greatly simplifies the build process because it is not necessary to build libffi a couple of times and then merge the resulting binaries.

I'm -0 at this point w.r.t. trying to remove libffi_osx, that tree works and I'll work on it anyway (it is also used in PyObjC).

Arfrever · 2013-02-13T16:41:40Z

Maybe Modules/_ctypes/libffi in 2.7, 3.2 and 3.3 branches should be updated?

merwok · 2013-02-13T23:20:12Z

Sound like a large change for stable releases. Usually we would not do that.

Arfrever · 2013-02-16T01:00:12Z

However internal copy of expat was updated in 2.7 and 3.2 branches about 2 weeks ago (issue bpo-14340).

gpshead · 2013-02-19T22:05:17Z

An update to libffi is needed for all maintained versions of Python.

In 2.7, we're running into the stack being misaligned in 32-bit x86 code which is something a libffi update fixes. It is a simple patch:

http://patchwork.ozlabs.org/patch/58128/

which comes to the official libffi releases via atgreen/libffi@3f5b137#src/x86/sysv.S.

The problem: without the stack being 16-byte aligned, code generated by modern compilers like recent gcc/g++ or clang assumed that the stack is 16 byte aligned and uses SSE instructions in some circumstances that require this. Without this fix, any ctypes call into such code will crash.

Sure, that tiny patch could be cherry picked into our libffi, but we IMNSHO may as well just update the entire thing given that we embed a very old copy rather than use it as an external dependency.

gpshead · 2013-02-19T22:11:59Z

http://bugs.python.org/issue17245 filed to track the stack alignment issue. The only reason i set this as release blocker is to let a release manager decide which of these two issues to proceed with for 2.7.4 and 3.2.4 (and 3.3.1).

benjaminp · 2013-02-19T22:12:40Z

By all means, upgrade it.

doko42 · 2013-02-19T22:29:51Z

before the updates, ... there seem to be two test failures on sparc solaris. the local libffi/src/sparc/v8.S change was integrated upstream, so I don't yet what could cause these failures. or did they fail before too?

birkenfeld · 2013-02-24T11:50:04Z

Updating 3.x branches is fine with me.

python-dev · 2013-03-18T23:59:15Z

New changeset b10ec5083a53 by Gregory P. Smith in branch '2.7':
Fixes issue bpo-17192: Update the ctypes module's libffi to v3.0.13. This
http://hg.python.org/cpython/rev/b10ec5083a53

python-dev · 2013-03-19T22:33:48Z

New changeset 43e8da639462 by Gregory P. Smith in branch '3.2':
Fixes issue bpo-17192: Update the ctypes module's libffi to v3.0.13. This
http://hg.python.org/cpython/rev/43e8da639462

New changeset f23274357fc8 by Gregory P. Smith in branch '3.3':
Fixes Issue bpo-17192: Update the ctypes module's libffi to v3.0.13. This
http://hg.python.org/cpython/rev/f23274357fc8

New changeset a94b3b4599f1 by Gregory P. Smith in branch 'default':
Fixes Issue bpo-17192: Update the ctypes module's libffi to v3.0.13. This
http://hg.python.org/cpython/rev/a94b3b4599f1

gpshead · 2013-03-19T23:13:59Z

2.7, 3.2, 3.3 and 3.4 have all be updated to ctypes v3.0.13.

fwiw, I tried a build of 3.3 on Sparc Solaris 10 using snakebite (nitrogen) and ctypes builds and links but two tests fail: ctypes.test.test_bitfields.C_Test test_ints and test_shorts failed.

That should be dealt with via a separate issue and by someone who cares about Solaris.

koobs · 2013-04-20T14:06:35Z

These break what was addressed in bpo-11729 for default, 3,x and 3.3.

2.7 seems to have made it through unscathed.

I'm not sure where or how the old code was introduced, but the clang fix has been upstreamed and is correct in the pure libffi 3.0.13 sources

Failure to build ctypes can be seen here:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.x/builds/1246/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.x/builds/1245/steps/test/logs/stdio
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.3/builds/538/steps/test/logs/stdio

birkenfeld · 2013-04-28T19:20:44Z

Greg?

gpshead · 2013-04-30T06:37:42Z

it _looks_ like the libffi updates i put in 3.2, 3.3 and default (3.4) are messed up with some changes missing in 3.2 and more missing in 3.3 and 3.4. (i'm comparing vs upstream 3.0.13 <-> 2.7 <-> 3.2 <-> 3.3 <-> 3.4).

i blame the mess of merging that was required by hg due to the plethora of versions of libffi we had in different states in 2.7, 3.2 and 3.3+ before i did the update. i'm fixing it now.

birkenfeld · 2013-04-30T06:59:06Z

Thanks. For 3.2, please attach a patch here.

gpshead · 2013-04-30T07:20:14Z

I've got this and all of the fixes necessary for 3.3, default and 2.7 in my local repositories.

odd thing in the 3.2 patch: it is entirely unclear to me if the bpo-10309 fix needs to be kept. i included it as it was missing from 3.2 after previous commits (3.3 and 3.4 still kept it). 2.7 needs it as well. the other part of the patch is the bpo-11729 fix that already exists upstream being reapplied; it was inadvertently undone during the upgrade.

I _could_ push fixes for all branches at once right now unless pushes to 3.2 are blocked and prevent me from pushing anything.

(rant: at which point i declare "hg hell" and feel like giving up because it simply has gotten in the way throughout this entire process rather than helping... i'll be forced to manually create diffs and reapply them to _other_ hg checkouts? not so useful. surely someone will stupidly claim i'm holding hg wrong instead of the tool being unhelpful).

We need to GET RID of libffi being bundled as part of the source release in 3.4. this is disgusting.

birkenfeld · 2013-04-30T07:31:56Z

OK, 3.2 block disabled, push away.

koobs · 2013-04-30T08:03:32Z

Ready to confirm on buildbots when this lands

Also, libffi-3.0.13 is installed on my buildbots if you want to use -custom to have a play, I can be pinged anytime on IRC if I can be of any assistance. koobs @ #python-dev

python-dev · 2013-04-30T08:05:38Z

New changeset 7a2e0c1bb1a9 by Gregory P. Smith in branch '3.2':

Fix bpo-17192 for 3.2 - reapply the bpo-11729 patch that was undone
http://hg.python.org/cpython/rev/7a2e0c1bb1a9

New changeset aa3371fa9773 by Gregory P. Smith in branch '3.3':

Fix PART of bpo-17192 for 3.3 - reapply the bpo-11729 patch
http://hg.python.org/cpython/rev/aa3371fa9773

New changeset e0fdc21c7d13 by Gregory P. Smith in branch 'default':

Fix PART of bpo-17192 for 3.4 - reapply the bpo-11729 patch
http://hg.python.org/cpython/rev/e0fdc21c7d13

New changeset c665abe15c4a by Gregory P. Smith in branch '3.3':

Fix remaining bits of bpo-17192 for 3.3 - these changes
http://hg.python.org/cpython/rev/c665abe15c4a

New changeset 688bc0b44d96 by Gregory P. Smith in branch 'default':

Fix remaining bits of bpo-17192 for 3.4 - these changes
http://hg.python.org/cpython/rev/688bc0b44d96

New changeset 663800e18fef by Gregory P. Smith in branch '3.2':
This local change was lost during the fixing of bpo-17192 to update
http://hg.python.org/cpython/rev/663800e18fef

New changeset 7438d202e380 by Gregory P. Smith in branch '2.7':
This local change was lost during the fixing of bpo-17192 to update
http://hg.python.org/cpython/rev/7438d202e380

doko42 added the extension-modules C modules in the Modules dir label Feb 12, 2013

gpshead added the release-blocker label Feb 19, 2013

gpshead self-assigned this Mar 18, 2013

Arfrever mannequin changed the title ~~libffi-3.0.12 import~~ libffi-3.0.13 import Mar 19, 2013

gpshead closed this as completed Mar 19, 2013

gpshead added the type-bug An unexpected behavior, bug, or error label Mar 19, 2013

bitdancer reopened this Apr 20, 2013

gpshead closed this as completed Apr 30, 2013

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libffi-3.0.13 import #61394

libffi-3.0.13 import #61394

doko42 commented Feb 12, 2013

doko42 commented Feb 12, 2013

python-dev mannequin commented Feb 12, 2013

ronaldoussoren commented Feb 12, 2013

Arfrever mannequin commented Feb 13, 2013

merwok commented Feb 13, 2013

Arfrever mannequin commented Feb 16, 2013

gpshead commented Feb 19, 2013

gpshead commented Feb 19, 2013

benjaminp commented Feb 19, 2013

doko42 commented Feb 19, 2013

birkenfeld commented Feb 24, 2013

python-dev mannequin commented Mar 18, 2013

python-dev mannequin commented Mar 19, 2013

gpshead commented Mar 19, 2013

koobs commented Apr 20, 2013

birkenfeld commented Apr 28, 2013

gpshead commented Apr 30, 2013

birkenfeld commented Apr 30, 2013

gpshead commented Apr 30, 2013

birkenfeld commented Apr 30, 2013

koobs commented Apr 30, 2013

python-dev mannequin commented Apr 30, 2013

libffi-3.0.13 import #61394

libffi-3.0.13 import #61394

Comments

doko42 commented Feb 12, 2013

doko42 commented Feb 12, 2013

python-dev mannequin commented Feb 12, 2013

ronaldoussoren commented Feb 12, 2013

Arfrever mannequin commented Feb 13, 2013

merwok commented Feb 13, 2013

Arfrever mannequin commented Feb 16, 2013

gpshead commented Feb 19, 2013

gpshead commented Feb 19, 2013

benjaminp commented Feb 19, 2013

doko42 commented Feb 19, 2013

birkenfeld commented Feb 24, 2013

python-dev mannequin commented Mar 18, 2013

python-dev mannequin commented Mar 19, 2013

gpshead commented Mar 19, 2013

koobs commented Apr 20, 2013

birkenfeld commented Apr 28, 2013

gpshead commented Apr 30, 2013

birkenfeld commented Apr 30, 2013

gpshead commented Apr 30, 2013

birkenfeld commented Apr 30, 2013

koobs commented Apr 30, 2013

python-dev mannequin commented Apr 30, 2013