classification
Title: Python doesn't build due to test_float.py broken on non-IEEE machines
Type: compile error Stage: resolved
Components: Installation Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: mark.dickinson Nosy List: mark.dickinson, r.david.murray, skrah, stark, tim.peters
Priority: normal Keywords:

Created on 2016-07-02 21:10 by stark, last changed 2016-08-29 15:10 by mark.dickinson. This issue is now closed.

Messages (9)
msg269739 - (view) Author: Greg Stark (stark) Date: 2016-07-02 21:10
The file Test/test_float.py has a decorator for tests that must be skipped on non-IEEE machines. However just eyeballing it it seems there are a few tests that test behaviour around Inf and NaN that are lacking this decorator.

Alternately it's possible this is actually a core interpreter bug where even code which is never called is now triggering floating point exceptions due to some constant propagation optimization or something like that.

To reproduce, try to install on a non-IEE machine such as a VAX and your install will crash with:

Compiling /usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/test/test_filecmp.py ...
Compiling /usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/test/test_fileinput.py ...
Compiling /usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/test/test_fileio.py ...
Compiling /usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/test/test_float.py ...
[1]   Floating point exception (core dumped) PYTHONPATH=/usr/...
*** Error code 136

Stop.
make: stopped in /usr/pkgsrc/lang/python27/work/Python-2.7.10
*** Error code 1


I do happen to have been compiling with -O0 -g so I have a core file and a binary with symbols (if you have a VAX gdb...) but here's the backtrace:

#0  0x7f687748 in float_mul (v=0x7ee82f54, w=0x7ee82f54, 2129145684, 2129145684) at Objects/floatobject.c:658
#1  0x7f65eb56 in binary_op1 (v=0x7ee82f54, w=0x7ee82f54, op_slot=8, 2129145684, 2129145684, 8) at Objects/abstract.c:945
#2  0x7f65f572 in PyNumber_Multiply (v=0x7ee82f54, w=0x7ee82f54, 2129145684, 2129145684) at Objects/abstract.c:1216
#3  0x7f72e573 in fold_binops_on_constants (codestr=0x7f28b169 "d\001", consts=0x7f0b5dcc, 2133373289, 2131451340)
    at Python/peephole.c:108
#4  0x7f72f56c in PyCode_Optimize (code=0x7f3316a8, consts=0x7f0b5dcc, names=0x7f0948ec, 
    lineno_obj=0x7f0b5840, 2134054568, 2131451340, 2131314924, 2131449920) at Python/peephole.c:482
#5  0x7f71d3ad in makecode (c=0x7fffc810, a=0x7fffc5b0, 2147469328, 2147468720) at Python/compile.c:3883
#6  0x7f71d77d in assemble (c=0x7fffc810, addNone=1, 2147469328, 1) at Python/compile.c:3999
#7  0x7f717527 in compiler_function (c=0x7fffc810, s=0x7eb82bf0, 2147469328, 2125999088) at Python/compile.c:1422
#8  0x7f719552 in compiler_visit_stmt (c=0x7fffc810, s=0x7eb82bf0, 2147469328, 2125999088) at Python/compile.c:2122
#9  0x7f716d00 in compiler_body (c=0x7fffc810, stmts=0x7ebc88f8, 2147469328, 2126285048) at Python/compile.c:1209
#10 0x7f717882 in compiler_class (c=0x7fffc810, s=0x7ebd2608, 2147469328, 2126325256) at Python/compile.c:1477
#11 0x7f719564 in compiler_visit_stmt (c=0x7fffc810, s=0x7ebd2608, 2147469328, 2126325256) at Python/compile.c:2124
#12 0x7f716d00 in compiler_body (c=0x7fffc810, stmts=0x7eb53010, 2147469328, 2125803536) at Python/compile.c:1209
#13 0x7f716db6 in compiler_mod (c=0x7fffc810, mod=0x7ebe1438, 2147469328, 2126386232) at Python/compile.c:1229
#14 0x7f7151f3 in PyAST_Compile (mod=0x7ebe1438, filename=0x7f2c15b4 "/usr/pkg/lib/python2.7/test/test_float.py", 
    flags=0x7fffc8c0, arena=0x7f3482f0, 2126386232, 2133595572, 2147469504, 2134147824) at Python/compile.c:292
#15 0x7f733849 in Py_CompileStringFlags (
    str=0x7eb43014 "\nimport unittest, struct\nimport os\nfrom test import test_support\nimport math\nfrom math import isinf, isnan, copysign, ldexp\nimport operator\nimport random\nimport fractions\nimport sys\nimport time\n\nINF ="..., 
    filename=0x7f2c15b4 "/usr/pkg/lib/python2.7/test/test_float.py", start=257, 
    flags=0x7fffc8c0, 2125738004, 2133595572, 257, 2147469504) at Python/pythonrun.c:1428
#16 0x7f705dc6 in builtin_compile (self=0x0, args=0x7f08f914, kwds=0x0, 0, 2131294484, 0) at Python/bltinmodule.c:559
#17 0x7f6a5b8a in PyCFunction_Call (func=0x7f32a96c, arg=0x7f08f914, kw=0x0, 2134026604, 2131294484, 0)
    at Objects/methodobject.c:85
#18 0x7f713023 in call_function (pp_stack=0x7fffca44, oparg=3, 2147469892, 3) at Python/ceval.c:4035
#19 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f29568c, throwflag=0, 2133415564, 0) at Python/ceval.c:2681
#20 0x7f711523 in PyEval_EvalCodeEx (co=0x7f25cad0, globals=0x7f29a3e4, locals=0x0, args=0x7f3b7874, argcount=4, 
    kws=0x7f3b7884, kwcount=0, defs=0x7f29d3a8, defcount=3, 
    closure=0x0, 2133183184, 2133435364, 0, 2134603892, 4, 2134603908, 0, 2133447592, 3, 0) at Python/ceval.c:3267
#21 0x7f71337c in fast_function (func=0x7f29b924, pp_stack=0x7fffcd30, n=4, na=4, nk=0, 2133440804, 2147470640, 4, 4, 0)
    at Python/ceval.c:4131
#22 0x7f713106 in call_function (pp_stack=0x7fffcd30, oparg=4, 2147470640, 4) at Python/ceval.c:4056
#23 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f3b76ec, throwflag=0, 2134603500, 0) at Python/ceval.c:2681
#24 0x7f711523 in PyEval_EvalCodeEx (co=0x7f23ecc8, globals=0x7f33e714, locals=0x0, args=0x7f2d1198, argcount=5, 
    kws=0x7f2d11ac, kwcount=0, defs=0x7f24bfd8, defcount=4, 
    closure=0x0, 2133060808, 2134107924, 0, 2133660056, 5, 2133660076, 0, 2133114840, 4, 0) at Python/ceval.c:3267
#25 0x7f71337c in fast_function (func=0x7f29b844, pp_stack=0x7fffd01c, n=5, na=5, nk=0, 2133440580, 2147471388, 5, 5, 0)
    at Python/ceval.c:4131
#26 0x7f713106 in call_function (pp_stack=0x7fffd01c, oparg=5, 2147471388, 5) at Python/ceval.c:4056
#27 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f2d102c, throwflag=0, 2133659692, 0) at Python/ceval.c:2681
#28 0x7f711523 in PyEval_EvalCodeEx (co=0x7f258188, globals=0x7f33e714, locals=0x0, args=0x7f295660, argcount=6, 
    kws=0x7f295678, kwcount=0, defs=0x7f24cc68, defcount=5, 
    closure=0x0, 2133164424, 2134107924, 0, 2133415520, 6, 2133415544, 0, 2133118056, 5, 0) at Python/ceval.c:3267
#29 0x7f71337c in fast_function (func=0x7f29b7d4, pp_stack=0x7fffd308, n=6, na=6, nk=0, 2133440468, 2147472136, 6, 6, 0)
    at Python/ceval.c:4131
#30 0x7f713106 in call_function (pp_stack=0x7fffd308, oparg=6, 2147472136, 6) at Python/ceval.c:4056
#31 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f2954f4, throwflag=0, 2133415156, 0) at Python/ceval.c:2681
#32 0x7f711523 in PyEval_EvalCodeEx (co=0x7f258188, globals=0x7f33e714, locals=0x0, args=0x7f2a31a8, argcount=6, 
    kws=0x7f2a31c0, kwcount=0, defs=0x7f24cc68, defcount=5, 
    closure=0x0, 2133164424, 2134107924, 0, 2133471656, 6, 2133471680, 0, 2133118056, 5, 0) at Python/ceval.c:3267
#33 0x7f71337c in fast_function (func=0x7f29b7d4, pp_stack=0x7fffd5f4, n=6, na=6, nk=0, 2133440468, 2147472884, 6, 6, 0)
    at Python/ceval.c:4131
#34 0x7f713106 in call_function (pp_stack=0x7fffd5f4, oparg=6, 2147472884, 6) at Python/ceval.c:4056
#35 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f2a302c, throwflag=0, 2133471276, 0) at Python/ceval.c:2681
#36 0x7f7132c0 in fast_function (func=0x7f29ba04, pp_stack=0x7fffd7e0, n=0, na=0, nk=0, 2133441028, 2147473376, 0, 0, 0)
    at Python/ceval.c:4121
#37 0x7f713106 in call_function (pp_stack=0x7fffd7e0, oparg=0, 2147473376, 0) at Python/ceval.c:4056
#38 0x7f70fe32 in PyEval_EvalFrameEx (f=0x7f25370c, throwflag=0, 2133145356, 0) at Python/ceval.c:2681
#39 0x7f711523 in PyEval_EvalCodeEx (co=0x7f25c0b0, globals=0x7f33e714, locals=0x7f33e714, args=0x0, argcount=0, kws=0x0, 
    kwcount=0, defs=0x0, defcount=0, closure=0x0, 2133180592, 2134107924, 2134107924, 0, 0, 0, 0, 0, 0, 0)
    at Python/ceval.c:3267
#40 0x7f70af1b in PyEval_EvalCode (co=0x7f25c0b0, globals=0x7f33e714, locals=0x7f33e714, 2133180592, 2134107924, 2134107924)
    at Python/ceval.c:669
#41 0x7f73365a in run_mod (mod=0x7f2f6e60, 
    filename=0x7fffef44 "/usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/compileall.py", globals=0x7f33e714, 
    locals=0x7f33e714, flags=0x7fffdbe4, 
    arena=0x7f348250, 2133814880, 2147479364, 2134107924, 2134107924, 2147474404, 2134147664) at Python/pythonrun.c:1371
#42 0x7f73360b in PyRun_FileExFlags (fp=0x7f5aa0c0, 
    filename=0x7fffef44 "/usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/compileall.py", start=257, 
    globals=0x7f33e714, locals=0x7f33e714, closeit=1, 
    flags=0x7fffdbe4, 2136645824, 2147479364, 257, 2134107924, 2134107924, 1, 2147474404) at Python/pythonrun.c:1357
#43 0x7f732658 in PyRun_SimpleFileExFlags (fp=0x7f5aa0c0, 
    filename=0x7fffef44 "/usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/compileall.py", closeit=1, 
    flags=0x7fffdbe4, 2136645824, 2147479364, 1, 2147474404) at Python/pythonrun.c:949
#44 0x7f73207a in PyRun_AnyFileExFlags (fp=0x7f5aa0c0, 
    filename=0x7fffef44 "/usr/pkgsrc/lang/python27/work/.destdir/usr/pkg/lib/python2.7/compileall.py", closeit=1, 
    flags=0x7fffdbe4, 2136645824, 2147479364, 1, 2147474404) at Python/pythonrun.c:753
#45 0x7f741fc2 in Py_Main (argc=11, argv=0x7fffdcdc, 11, 2147474652) at Modules/main.c:640
#46 0x00010742 in main (argc=11, argv=0x7fffdcdc, 11, 2147474652, 2147474700) at Modules/python.c:23
msg269746 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-07-03 00:01
It looks like you are in the compiling step, so the decorators have nothing to do with it.  This does appear to be a platform issue in the interpreter core, and you'll have to track it down since we don't have access to your platform (at least, I don't think any of the core devs are set up to test on VAX).
msg269762 - (view) Author: Greg Stark (stark) Date: 2016-07-03 11:43
I can easily grant you access to mine. It's a bit slow but you can quickly poke around with the state in the backtrace which honestly looks to me like it should be enough to spot the problem.

There are other options. There's a good emulator at http://simh.trailing-edge.com/ and there are other non-ieee platforms out there. But installing an OS is a bit of a hassle.

If you're interested send me your email address and preferably a public key or something and I'll send you a login.
msg269769 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-07-03 14:37
I don't have time to tackle it myself; maybe Mark will be interested, but he probably doesn't have time either.  Also, he's been known to say he'd like to drop support for non-IEEE architectures ;)  I suspect it will be up to someone with an investment in the platform, this being a volunteer project.
msg269788 - (view) Author: Greg Stark (stark) Date: 2016-07-04 15:33
I certainly understand the limitations of volunteer projects. I know you have limited resources and can't test on every platform. That's actually exactly why I'm testing on this platform and why I reported the bug. If there's any additional information I can provide to help you I can try to get it.

I found this while trying to build a Postgres build-farm member to help us identify portability issues quickly on platforms that aren't widely used by developers. If Python has similar infrastructure I could deploy it to the same hardware to help you spot these problems sooner.
msg269789 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2016-07-04 15:46
It looks like the peephole optimizer chokes on some constant folding.
Probably:

INF = float("inf")
NAN = float("nan")

-INF
-NAN

You could try some combinations in the REPL.
msg269814 - (view) Author: Greg Stark (stark) Date: 2016-07-04 23:32
Well I was able to minimize the actual cause:

$ /usr/pkg/bin/python -c 1e300*1e300
[1]   Floating point exception /usr/pkg/bin/python -c 1e300*1e300

I noticed that the constant folding arithmetic is protected by "PyFPE_START_PROTECT" macros so I wonder... should I be configuring with --with-fpectl? (Or should it be on unconditionally if it's required to function properly on a given architecture?)
msg269815 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-07-05 08:01
[David]

> maybe Mark will be interested, but he probably doesn't have time either.  Also, he's been known to say he'd like to drop support for non-IEEE architectures ;)

Exactly correct on all counts. :-) I'm *very* interested: I've been looking for a non-IEEE machine to play with Python on for years. But time is (as ever) in short supply.

Realistically, maintaining support for non-IEEE 754 platforms looks like something with an enormously high cost/benefit ratio; maintaining just on machines with (some approximation of) IEEE 754 is already enough work.

Greg: you say "there are other non-ieee platforms out there". Can you point me to any current hardware using non-IEEE 754 floating-point formats that one might want to run Python on? I know there are examples where not all of the IEEE 754 standard is supported, but the interchange format used is still IEEE 754 binary64 / binary32 / whatever, but that's not what I'm looking for; I'm looking for cases where the underlying format is different. The only example I'm aware of is the IBM System z, with its hexadecimal floating-point format, but those machines support IEEE 754 too.
msg273864 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-08-29 15:10
In the absence of a concrete use-case, I'm closing as "wont fix". Without suitable infrastructure (non-IEEE 754 buildbots, for example) and a clear need for this, the effort involved in maintaining CPython on non-IEEE 754 systems far outweighs the benefits.
History
Date User Action Args
2016-08-29 15:10:19mark.dickinsonsetstage: resolved
2016-08-29 15:10:08mark.dickinsonsetstatus: open -> closed
assignee: mark.dickinson
resolution: wont fix
messages: + msg273864
2016-07-05 08:01:28mark.dickinsonsetmessages: + msg269815
2016-07-04 23:32:45starksetmessages: + msg269814
2016-07-04 15:46:41skrahsetnosy: + skrah
messages: + msg269789
2016-07-04 15:33:55starksetmessages: + msg269788
2016-07-03 14:37:56r.david.murraysetmessages: + msg269769
2016-07-03 11:43:54starksetmessages: + msg269762
2016-07-03 00:01:27r.david.murraysetnosy: + r.david.murray
messages: + msg269746
2016-07-02 21:50:56rhettingersetnosy: + tim.peters, mark.dickinson
2016-07-02 21:10:51starkcreate