classification
Title: 3.0 make test failures on Solaris 10
Type: Stage:
Components: Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mark.dickinson, r.david.murray, terry.reedy
Priority: normal Keywords:

Created on 2008-12-03 21:00 by skip.montanaro, last changed 2011-07-22 19:44 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
cmath_debug.patch mark.dickinson, 2008-12-03 21:30 review
cmathmodule.S skip.montanaro, 2008-12-04 16:38 c_exp without extra printf() calls
cmathmodule.S.printf skip.montanaro, 2008-12-04 16:39 c_exp with extra printf() calls
Messages (18)
msg76839 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-03 21:00
I downloaded the 3.0 tarfile and did a straightforward

    configure
    make
    make test

on Solaris 10 and got several test failures:

    290 tests OK.
    4 tests failed:
        test_cmath test_math test_posix test_subprocess

Here's the output for just the failing tests:

    test test_cmath failed -- Traceback (most recent call last):
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_cmath.py", line 336,
      in test_specific_values
        self.fail('OverflowError not raised in test %s' % test_str)
    AssertionError: OverflowError not raised in test exp0052: exp(complex(710.0,
    1.5))

    test test_math failed -- errors occurred; run in verbose mode for details

    test test_posix failed -- Traceback (most recent call last):
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_posix.py", line 252,
      in test_getcwd_long_pathnames
        support.rmtree(base_path)
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/support.py", line 98, in
      rmtree    shutil.rmtree(path)
      File "/home/tuba/skipm/src/Python-3.0/Lib/shutil.py", line 225, in rmtree
        onerror(os.rmdir, path, sys.exc_info())
      File "/home/tuba/skipm/src/Python-3.0/Lib/shutil.py", line 223, in rmtree
        os.rmdir(path)
    OSError: [Errno 22] Invalid argument:
    '/home/tuba/skipm/src/Python-3.0/@test.getcwd'

    Could not find platform independent libraries <prefix>
    Could not find platform dependent libraries <exec_prefix>
    Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
    Fatal Python error: Py_Initialize: can't initialize sys standard streams
    ImportError: No module named encodings.utf_8
    .
        this bit of output is from a test of stdout in a different process ...
    test test_subprocess failed -- Traceback (most recent call last):
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_subprocess.py", line
      115, in test_executable
        self.assertEqual(p.returncode, 47)
    AssertionError: -6 != 47

Here's the test_math output run through regrtest with the -v option:

    ======================================================================
    FAIL: testLog (test.test_math.MathTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_math.py", line 510, in
      testLog
        self.assertRaises(ValueError, math.log, NINF)
    AssertionError: ValueError not raised by log

    ======================================================================
    FAIL: testLog10 (test.test_math.MathTests)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_math.py", line 532, in
      testLog10
        self.assertRaises(ValueError, math.log10, NINF)
    AssertionError: ValueError not raised by log10

    ----------------------------------------------------------------------
    Ran 39 tests in 0.294s

Skip
msg76840 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-03 21:07
I think you brought up the math and cmath errors before, and I never 
managed to get to the bottom of the problem.  I'll have another go.

I don't think the (c)math test failures should be regarded as terribly 
serious, though;  the tests are quite ridiculously strict.

Could you please tell me what

cmath.exp(complex(710.0, 1.5))

actually returns, if it doesn't raise OverflowError?

(Don't know nuffin about posix and subprocess.)
msg76841 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-03 21:13
Mark> I think you brought up the math and cmath errors before, and I
    Mark> never managed to get to the bottom of the problem.  I'll have
    Mark> another go.

I vaguely remember something about that.  If I can be a "test mule" for you,
let me know.

    Mark> Could you please tell me what

    Mark> cmath.exp(complex(710.0, 1.5))

    Mark> actually returns, if it doesn't raise OverflowError?

(1.5802653829857376e+307+inf*j)

Maybe it needs to overflow along both axes???

S
msg76843 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-03 21:30
> (1.5802653829857376e+307+inf*j)

Those values look right;  except that there's some code near the end of 
the cexp function that's supposed to set errno to ERANGE if either the 
real or imaginary component of the result is infinity (and then math_1 
knows to raise OverflowError as a result).  It looks like that's not 
happening for some reason.

If you have time, could you try the attached patch and report what gets 
printed when cmath.exp(710+1.5j) is called?  On my machine, I get:

Python 3.1a0 (py3k:67510M, Dec  3 2008, 20:56:19) 
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmath
>>> cmath.exp(710.0 + 1.5j)
r.real, r.imag: 1.58027e+307, inf
Py_IS_INFINITY(r.real), Py_IS_INFINITY(r.imag): 0, 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: math range error

Mark
msg76845 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-03 21:43
Tracker issue:  I don't seem to be able to remove the 'patch' keyword
without (accidentally) ending up with something else---in this case the
64bit keyword.  Is this just me being incompetent, or should I file a
tracker bug?
msg76890 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-04 15:49
Mark> If you have time, could you try the attached patch and report what
    Mark> gets printed when cmath.exp(710+1.5j) is called?  On my machine, I
    Mark> get:
    ...

Looks similar here:

    % ./python
    Python 3.0 (r30:67503, Dec  3 2008, 14:46:39)
    [GCC 4.2.2] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import cmath
    >>> cmath.exp(complex(710.0, 1.5))
    r.real, r.imag: 1.58027e+307, Inf
    Py_IS_INFINITY(r.real), Py_IS_INFINITY(r.imag): 0, 1
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OverflowError: math range error

If I then comment back out your two printf statements and recompile, it once
again fails to raise OverflowError:

    % ./python
    Python 3.0 (r30:67503, Dec  3 2008, 14:46:39)
    [GCC 4.2.2] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import cmath ; cmath.exp(complex(710.0, 1.5))
    (1.5802653829857376e+307+inf*j)

Very weird.  If I uncomment either of your printf statements OverflowError
seems to be raised as expected.  It's as if some register bit isn't being
set unless you reference r.real or r.imag multiple times.

Skip
msg76894 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-04 16:38
Mark,

I trimmed down cmathmodule.c to just contain c_exp then 
generated assembler files for the non-printf and printf
cases.  Perhaps that will help you see what's going on.

Skip
msg76929 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-04 21:23
Thanks for the assembly code---you're running Solaris on x86!  Why 
didn't you say so before? :)

I think I have an idea what's going on:  it's the old extended-precision 
versus double-precision problem.  The calculation of c_exp is done in 
extended precision in the 80-bit registers of the x87 FPU.  The 
imaginary part of the result of c_exp(710+1.5j) is representable in 
extended precision, but is outside the range of double precision.   So 
at the point of the Py_IS_INFINITY call, the comparison is done in the 
FPU and the value being tested isn't infinity.  But sometime after that 
this value is forced out of the 80-bit extended precision FPU register 
and into memory, where it becomes a 64-bit IEEE 754 double precision 
infinity.

I guess the printf calls force the value from register to memory 
earlier, so that by the time of the Py_IS_INFINITY call it's already a 
64-bit double, and hence already an infinity.

Now to prove my theory by turning this into a fix, somehow.

It seems as though this problem isn't really specific to Solaris;  I 
guess it's just luck that it hasn't shown up on other x86 platforms.  It 
should certainly be fixed.


For the test_math failure, there have been problems with log on other 
platforms, too, mostly due to strange libm behaviour.  The obvious 
solution is to adapt the Python implementation to deal with special 
values itself, rather than leaving them to the platform math library.  
Should be a fairly straightforward change.
msg76933 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-04 21:54
Mark> Thanks for the assembly code---you're running Solaris on x86!  Why
    Mark> didn't you say so before? :)

I'm failry sure I can find a SPARC here to run it on as well.  They are
rather few and far between though.

Skip
msg77003 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-05 11:18
> I'm failry sure I can find a SPARC here to run it on as well.  They
> are rather few and far between though.

I don't think that's necessary.  Thanks for disabusing me of my 'Solaris
implies SPARC' mindset, though!

There are two more pieces of information that *would* be useful:

1. What happens if you build with the '-ffloat-store' option to gcc?
If my diagnosis is correct I'd expect the cmath tests to pass with this
flag.  (I'm not 100% sure how to make sure the '-ffloat-store' option
gets passed through into the Module builds, though
"BASECFLAGS='-ffloat-store' ./configure" seems to work for me.)

2. It looks as though the configure script isn't finding 'isinf' on
Solaris 10.  Any ideas why?  Is there some replacement for isinf that's
spelt differently?  I suspect that if we were using the system lib's
test for infinity instead of the Py_IS_INFINITY workaround then this
problem wouldn't have come up.

Mark
msg77035 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-05 15:54
Mark> 1. What happens if you build with the '-ffloat-store' option to
    Mark>    gcc?

Doesn't quite work:

    % ./python
    Python 3.0 (r30:67503, Dec  5 2008, 09:48:42)
    [GCC 4.2.2] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import cmath
    >>> cmath.exp(complex(710.0, 1.5))
    Segmentation Fault (core dumped)

:-/

    Mark> 2. It looks as though the configure script isn't finding 'isinf'
    Mark>    on Solaris 10.  Any ideas why?  Is there some replacement for
    Mark>    isinf that's spelt differently?  I suspect that if we were
    Mark>    using the system lib's test for infinity instead of the
    Mark>    Py_IS_INFINITY workaround then this problem wouldn't have come
    Mark>    up.

Thanks for the tip.  The configure script doesn't #include <math.h> so
isinf() is not mapped to __builtin_isinf().  Consequently the conftest link
fails:

    configure:21401: checking for isinf
    configure:21457: gcc -o conftest -g -O2   conftest.c -lresolv -lsocket -lnsl
    -lrt -ldl  -lm >&5
    conftest.c:252: warning: conflicting types for built-in function 'isinf'
    Undefined                        first referenced
     symbol                                                 in file
    isinf                               /var/tmp//ccmTAet6.o
    ld: fatal: Symbol referencing errors. No output written to conftest
    collect2: ld returned 1 exit status

I found this code in /usr/include/iso/math_c99.h, included by <math.h>:

    #undef      isinf
    #define     isinf(x)        __builtin_isinf(x)

Skip
msg77140 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-06 15:58
The segfault is a little worrying;  I don't understand that at all.

Skip, can you come up with a configure patch that would allow isinf to be 
detected on Solaris?

I'll also patch Py_IS_INFINITY to make sure that it forces its argument 
into memory before testing it;  this *should* fix the problem on platforms 
that don't have isinf.  (Note to self:  first find out whether 
Py_IS_INFINITY is ever applied to single precision floats, or whether we 
can assume the argument is always a double.)
msg77162 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-12-06 19:03
> can you come up with a configure patch that would allow isinf to be
> detected on Solaris?

The plot thickens.  I know squat about autoconf sorts of things so I
asked on the autoconf mailing list.  Eric Drake responded (in part):

  The Python ACHECK_FUNCS test should be rewritten (with proper m4
  quoting) as:

  AC_CHECK_FUNCS([acosh asinh atanh expm1 finite log1p])
  AC_CHECK_DECLS([isinf, isnan], [], [], [[#include <math.h>]])

so I gave that a whirl.  Whaddya know?  isinf *really* isn't available,
at least not until C99 apparently.  The AC_CHECK_DECLS macro generates
a conftest.c which looks like this:

  #include <math.h> 
 
  int 
  main () 
  { 
  #ifndef isinf 
    (void) isinf; 
  #endif 
 
    ; 
    return 0; 
  } 

which fails to compile/link on our Sol10 boxes.  Turns out libm.so
doesn't export an _isinf symbol.  The macro I found (in iso/math_c99.h)
doesn't expand in gcc -E output of the above code, so in my non-c99
environment I don't get to use isinf.  Looks like your Py_IS_INFINITY
fix will be necessary.  Eric also suggested using the gnu isinf and
isnan modules:

  http://git.savannah.gnu.org/gitweb/?
p=gnulib.git;a=blob;f=m4/isinf.m4;h=1b9e45a;hb=67461c3

I'll leave that for you to decide.

In any case, the above changes to AC_CHECK_FUNCS should probably be
made.

Skip
msg77224 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2008-12-07 12:50
Thanks, Skip.  It looks like the top priority is fixing Py_IS_INFINITY, 
then (still assuming that I'm not barking up entirely the wrong tree).  
I've opened issue 4575 for the Py_IS_INFINITY fix.

I'll look at the changes to AC_CHECK_FUNCS, too.
msg79065 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-04 15:26
autoconf checks for isinf and isnan fixed in r68299.
I also added a check for isfinite, which should really be used in 
preference to finite:  isfinite is standard in C99, while finite doesn't 
seem to be part of any standard.
msg90353 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-09 20:31
NB: I fixed the test_posix failure on trunk/2.6/py3k/3.1 in r73908,
r73914, r73913, and r73915.
msg123973 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-14 19:14
Are there any open problems left here or can this bug be closed?
msg140902 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-07-22 19:44
This appears to be fixed.

Skip: keywords now has a '-no selection-' option to get rid of keywords
History
Date User Action Args
2011-07-22 19:44:01terry.reedysetstatus: open -> closed

nosy: + terry.reedy
messages: + msg140902

keywords: - 64bit
resolution: fixed
2010-12-14 19:14:06r.david.murraysetmessages: + msg123973
2010-05-20 20:30:41skip.montanarosetnosy: - skip.montanaro
2009-07-09 20:31:52r.david.murraysetnosy: + r.david.murray
messages: + msg90353
2009-01-04 15:26:03mark.dickinsonsetmessages: + msg79065
2008-12-07 12:50:16mark.dickinsonsetmessages: + msg77224
2008-12-06 19:03:44skip.montanarosetmessages: + msg77162
2008-12-06 15:58:02mark.dickinsonsetmessages: + msg77140
2008-12-05 15:54:19skip.montanarosetmessages: + msg77035
2008-12-05 11:18:37mark.dickinsonsetmessages: + msg77003
2008-12-04 21:54:51skip.montanarosetmessages: + msg76933
2008-12-04 21:23:23mark.dickinsonsetmessages: + msg76929
2008-12-04 16:39:02skip.montanarosetfiles: + cmathmodule.S.printf
2008-12-04 16:38:41skip.montanarosetfiles: + cmathmodule.S
messages: + msg76894
2008-12-04 15:49:41skip.montanarosetmessages: + msg76890
2008-12-03 21:43:22mark.dickinsonsetmessages: + msg76845
2008-12-03 21:35:24mark.dickinsonsetkeywords: + 64bit, - patch
2008-12-03 21:30:24mark.dickinsonsetfiles: + cmath_debug.patch
keywords: + patch
messages: + msg76843
2008-12-03 21:13:36skip.montanarosetmessages: + msg76841
2008-12-03 21:07:58mark.dickinsonsetnosy: + mark.dickinson
messages: + msg76840
2008-12-03 21:00:23skip.montanarocreate