Hi!

As a followup to http://bugs.python.org/issue17237 I took the tree of Python 3.3.1 I had compiled already, with the patch that was eventually committed for issue17237, and ran the testsuite. Since I used configure flags to get it compiled quickly, it probably lacks functionality that’s normally there, so some tests got omitted.

The result of normally running the testsuite is attached as file; I can re-run any test verbosely, just ask ;-)

I can immediately say something to some failures:

① On m68k, the 68881 FPU, similar to the i8087 FPU, internally operates on 80-bit precision, which means that the “old” dtoa code is used. Contrarily to i8087, we do not want to disable that mode because that is not fully supported in some environments; especially, at least the most wide-spread emulator had bugs wrt. that. (Do note that, when you disable 80-bit precision on i8087, you get double-rounding as the only the mantisse, not the exponent, is bit-reduced, anyway. So I believe that you should not restrict the i8087 to 64-bit precision either.)

That’s responsible for test_optparse, test_difflib, maybe test_cmath, definitely test_format, test_types, test_math… ouch, quite some.

It would be interesting to see what happens when you take a normal i386 (not amd64 or x32) Linux host and compile py3k with SSE disabled (to force it to use the normal i80387 FPU), and disable HAVE_GCC_ASM_FOR_X87 so that X87_DOUBLE_ROUNDING is defined, because that’s what we have on m68k, and see whether the failure mode is the same. If so, you can then probably relatively easily fix those.

root@ara3:~/P-without-block # fgrep X87 pyconfig.h                                                              
/* #undef HAVE_GCC_ASM_FOR_X87 */
#define X87_DOUBLE_ROUNDING 1

② This one is pretty severe: AssertionError: 42 != 44 : wrong size for <class 'BaseException'>: got 42, expected 44

On m68k, “natural” alignment is *not* used: instead of aligning 4-byte quantities on 4-byte boundaries, for almost all types, only a minimum alignment of 2 bytes is used; this also has effect on struct padding, array padding, and things like that.

As shown in https://mail.gnome.org/archives/commits-list/2012-February/msg02332.html fixing this is a “simple” matter of making these invalid assumptions explicit. To that effect, take Include/pyerrors.h and change
#define PyException_HEAD PyObject_HEAD PyObject *dict;\
             PyObject *args; PyObject *traceback;\
             PyObject *context; PyObject *cause;\
             char suppress_context;
to
#define PyException_HEAD PyObject_HEAD PyObject *dict;\
             PyObject *args; PyObject *traceback;\
             PyObject *context; PyObject *cause;\
             char suppress_context; char padding1[3];
i.e. add the padding that’s implicit on “natural alignment” platforms (a padding1[1] is implicit on m68k). Do *not* use -malign-int because that changes the ABI, and you probably won’t be able to make libc or syscalls any more…

However, this brings us to a problem with that solution (which I’d still prefer over everything else) in Debian (hence, putting doko on Cc), which I’ll demonstrate on the shortest struct I could find in that file, but applies to e.g. PyImportErrorObject too:

typedef struct {
    PyException_HEAD 
    PyObject *code;  
} PySystemExitObject;

The “code” member of this struct is aligned with only *one* byte of padding, instead of three. This means that, when we apply this bugfix, that the published ABI changes: currently, sizeof(PySystemExitObject) is (8 (PyObject_HEAD) + 4 (dict) + 4 (args) + 4 (traceback) + 4 (context) + 4 (cause) + 1 (suppress_context) + 1 padding + 4 (code)) = 34; after the fact, it will be 36, and the offsetof(code) will have changed.

I’ll leave the understanding of these implications, and how they can best be handled in the distribution, to doko, because he’s probably the man who understands it best. I guess we need versioned dependencies or ABI virtual dependencies, since python3.3 is already uploaded, but some packages have already been built against it (probably not too difficult to identify them).

Or we can only get this fixed in python3.4 – no idea when that will land in Debian, or whether it’ll become the default python3 provider shortly.

③ As for any other failures, I don’t directly have an idea, but I guess we can work on them. I’d somewhat prefer to do that *after* the FPU precision-related bugs have been fixed, though, as it’ll reduce the noise, and possibly followup-errors. (Maybe I should even, for running the tests, fix the alignment issue locally in the tree.)