Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.0 make test failures on Solaris 10 #48756

Closed
smontanaro opened this issue Dec 3, 2008 · 18 comments
Closed

3.0 make test failures on Solaris 10 #48756

smontanaro opened this issue Dec 3, 2008 · 18 comments

Comments

@smontanaro
Copy link
Contributor

BPO 4506
Nosy @terryjreedy, @mdickinson, @bitdancer
Files
  • cmath_debug.patch
  • cmathmodule.S: c_exp without extra printf() calls
  • cmathmodule.S.printf: c_exp with extra printf() calls
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2011-07-22.19:44:01.247>
    created_at = <Date 2008-12-03.21:00:23.440>
    labels = []
    title = '3.0 make test failures on Solaris 10'
    updated_at = <Date 2011-07-22.19:44:01.230>
    user = 'https://github.com/smontanaro'

    bugs.python.org fields:

    activity = <Date 2011-07-22.19:44:01.230>
    actor = 'terry.reedy'
    assignee = 'none'
    closed = True
    closed_date = <Date 2011-07-22.19:44:01.247>
    closer = 'terry.reedy'
    components = []
    creation = <Date 2008-12-03.21:00:23.440>
    creator = 'skip.montanaro'
    dependencies = []
    files = ['12211', '12219', '12220']
    hgrepos = []
    issue_num = 4506
    keywords = []
    message_count = 18.0
    messages = ['76839', '76840', '76841', '76843', '76845', '76890', '76894', '76929', '76933', '77003', '77035', '77140', '77162', '77224', '79065', '90353', '123973', '140902']
    nosy_count = 3.0
    nosy_names = ['terry.reedy', 'mark.dickinson', 'r.david.murray']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue4506'
    versions = []

    @smontanaro
    Copy link
    Contributor Author

    I downloaded the 3.0 tarfile and did a straightforward

    configure
    make
    make test
    

    on Solaris 10 and got several test failures:

    290 tests OK.
    4 tests failed:
        test_cmath test_math test_posix test_subprocess
    

    Here's the output for just the failing tests:

        test test_cmath failed -- Traceback (most recent call last):
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_cmath.py", line 336,
          in test_specific_values
            self.fail('OverflowError not raised in test %s' % test_str)
        AssertionError: OverflowError not raised in test exp0052: exp(complex(710.0,
        1.5))
    test test_math failed -- errors occurred; run in verbose mode for details
    
        test test_posix failed -- Traceback (most recent call last):
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_posix.py", line 252,
          in test_getcwd_long_pathnames
            support.rmtree(base_path)
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/support.py", line 98, in
          rmtree    shutil.rmtree(path)
          File "/home/tuba/skipm/src/Python-3.0/Lib/shutil.py", line 225, in rmtree
            onerror(os.rmdir, path, sys.exc_info())
          File "/home/tuba/skipm/src/Python-3.0/Lib/shutil.py", line 223, in rmtree
            os.rmdir(path)
        OSError: [Errno 22] Invalid argument:
        '/home/tuba/skipm/src/Python-3.0/@test.getcwd'
    
        Could not find platform independent libraries <prefix>
        Could not find platform dependent libraries <exec_prefix>
        Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
        Fatal Python error: Py_Initialize: can't initialize sys standard streams
        ImportError: No module named encodings.utf_8
        .
            this bit of output is from a test of stdout in a different process ...
        test test_subprocess failed -- Traceback (most recent call last):
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_subprocess.py", line
          115, in test_executable
            self.assertEqual(p.returncode, 47)
        AssertionError: -6 != 47

    Here's the test_math output run through regrtest with the -v option:

    \======================================================================
    FAIL: testLog (test.test_math.MathTests)
    \----------------------------------------------------------------------
    
        Traceback (most recent call last):
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_math.py", line 510, in
          testLog
            self.assertRaises(ValueError, math.log, NINF)
        AssertionError: ValueError not raised by log
    \======================================================================
    FAIL: testLog10 (test.test_math.MathTests)
    \----------------------------------------------------------------------
    
        Traceback (most recent call last):
          File "/home/tuba/skipm/src/Python-3.0/Lib/test/test_math.py", line 532, in
          testLog10
            self.assertRaises(ValueError, math.log10, NINF)
        AssertionError: ValueError not raised by log10
    \----------------------------------------------------------------------
    Ran 39 tests in 0.294s
    

    Skip

    @mdickinson
    Copy link
    Member

    I think you brought up the math and cmath errors before, and I never
    managed to get to the bottom of the problem. I'll have another go.

    I don't think the (c)math test failures should be regarded as terribly
    serious, though; the tests are quite ridiculously strict.

    Could you please tell me what

    cmath.exp(complex(710.0, 1.5))

    actually returns, if it doesn't raise OverflowError?

    (Don't know nuffin about posix and subprocess.)

    @smontanaro
    Copy link
    Contributor Author

    Mark> I think you brought up the math and cmath errors before, and I
    Mark> never managed to get to the bottom of the problem. I'll have
    Mark> another go.

    I vaguely remember something about that. If I can be a "test mule" for you,
    let me know.

    Mark> Could you please tell me what
    
    Mark> cmath.exp(complex(710.0, 1.5))
    
    Mark> actually returns, if it doesn't raise OverflowError?
    

    (1.5802653829857376e+307+inf*j)

    Maybe it needs to overflow along both axes???

    S

    @mdickinson
    Copy link
    Member

    (1.5802653829857376e+307+inf*j)

    Those values look right; except that there's some code near the end of
    the cexp function that's supposed to set errno to ERANGE if either the
    real or imaginary component of the result is infinity (and then math_1
    knows to raise OverflowError as a result). It looks like that's not
    happening for some reason.

    If you have time, could you try the attached patch and report what gets
    printed when cmath.exp(710+1.5j) is called? On my machine, I get:

    Python 3.1a0 (py3k:67510M, Dec  3 2008, 20:56:19) 
    [GCC 4.0.1 (Apple Inc. build 5490)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import cmath
    >>> cmath.exp(710.0 + 1.5j)
    r.real, r.imag: 1.58027e+307, inf
    Py_IS_INFINITY(r.real), Py_IS_INFINITY(r.imag): 0, 1
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OverflowError: math range error

    Mark

    @mdickinson
    Copy link
    Member

    Tracker issue: I don't seem to be able to remove the 'patch' keyword
    without (accidentally) ending up with something else---in this case the
    64bit keyword. Is this just me being incompetent, or should I file a
    tracker bug?

    @smontanaro
    Copy link
    Contributor Author

    Mark> If you have time, could you try the attached patch and report what
    Mark> gets printed when cmath.exp(710+1.5j) is called? On my machine, I
    Mark> get:
    ...

    Looks similar here:

        % ./python
        Python 3.0 (r30:67503, Dec  3 2008, 14:46:39)
        [GCC 4.2.2] on sunos5
        Type "help", "copyright", "credits" or "license" for more information.
        >>> import cmath
        >>> cmath.exp(complex(710.0, 1.5))
        r.real, r.imag: 1.58027e+307, Inf
        Py_IS_INFINITY(r.real), Py_IS_INFINITY(r.imag): 0, 1
        Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
        OverflowError: math range error

    If I then comment back out your two printf statements and recompile, it once
    again fails to raise OverflowError:

        % ./python
        Python 3.0 (r30:67503, Dec  3 2008, 14:46:39)
        [GCC 4.2.2] on sunos5
        Type "help", "copyright", "credits" or "license" for more information.
        >>> import cmath ; cmath.exp(complex(710.0, 1.5))
        (1.5802653829857376e+307+inf*j)

    Very weird. If I uncomment either of your printf statements OverflowError
    seems to be raised as expected. It's as if some register bit isn't being
    set unless you reference r.real or r.imag multiple times.

    Skip

    @smontanaro
    Copy link
    Contributor Author

    Mark,

    I trimmed down cmathmodule.c to just contain c_exp then
    generated assembler files for the non-printf and printf
    cases. Perhaps that will help you see what's going on.

    Skip

    @mdickinson
    Copy link
    Member

    Thanks for the assembly code---you're running Solaris on x86! Why
    didn't you say so before? :)

    I think I have an idea what's going on: it's the old extended-precision
    versus double-precision problem. The calculation of c_exp is done in
    extended precision in the 80-bit registers of the x87 FPU. The
    imaginary part of the result of c_exp(710+1.5j) is representable in
    extended precision, but is outside the range of double precision. So
    at the point of the Py_IS_INFINITY call, the comparison is done in the
    FPU and the value being tested isn't infinity. But sometime after that
    this value is forced out of the 80-bit extended precision FPU register
    and into memory, where it becomes a 64-bit IEEE 754 double precision
    infinity.

    I guess the printf calls force the value from register to memory
    earlier, so that by the time of the Py_IS_INFINITY call it's already a
    64-bit double, and hence already an infinity.

    Now to prove my theory by turning this into a fix, somehow.

    It seems as though this problem isn't really specific to Solaris; I
    guess it's just luck that it hasn't shown up on other x86 platforms. It
    should certainly be fixed.

    For the test_math failure, there have been problems with log on other
    platforms, too, mostly due to strange libm behaviour. The obvious
    solution is to adapt the Python implementation to deal with special
    values itself, rather than leaving them to the platform math library.
    Should be a fairly straightforward change.

    @smontanaro
    Copy link
    Contributor Author

    Mark> Thanks for the assembly code---you're running Solaris on x86! Why
    Mark> didn't you say so before? :)

    I'm failry sure I can find a SPARC here to run it on as well. They are
    rather few and far between though.

    Skip

    @mdickinson
    Copy link
    Member

    I'm failry sure I can find a SPARC here to run it on as well. They
    are rather few and far between though.

    I don't think that's necessary. Thanks for disabusing me of my 'Solaris
    implies SPARC' mindset, though!

    There are two more pieces of information that *would* be useful:

    1. What happens if you build with the '-ffloat-store' option to gcc?
      If my diagnosis is correct I'd expect the cmath tests to pass with this
      flag. (I'm not 100% sure how to make sure the '-ffloat-store' option
      gets passed through into the Module builds, though
      "BASECFLAGS='-ffloat-store' ./configure" seems to work for me.)

    2. It looks as though the configure script isn't finding 'isinf' on
      Solaris 10. Any ideas why? Is there some replacement for isinf that's
      spelt differently? I suspect that if we were using the system lib's
      test for infinity instead of the Py_IS_INFINITY workaround then this
      problem wouldn't have come up.

    Mark

    @smontanaro
    Copy link
    Contributor Author

    Mark> 1. What happens if you build with the '-ffloat-store' option to
    Mark> gcc?

    Doesn't quite work:

        % ./python
        Python 3.0 (r30:67503, Dec  5 2008, 09:48:42)
        [GCC 4.2.2] on sunos5
        Type "help", "copyright", "credits" or "license" for more information.
        >>> import cmath
        >>> cmath.exp(complex(710.0, 1.5))
        Segmentation Fault (core dumped)

    :-/

    Mark> 2. It looks as though the configure script isn't finding 'isinf'
    Mark>    on Solaris 10.  Any ideas why?  Is there some replacement for
    Mark>    isinf that's spelt differently?  I suspect that if we were
    Mark>    using the system lib's test for infinity instead of the
    Mark>    Py_IS_INFINITY workaround then this problem wouldn't have come
    Mark>    up.
    

    Thanks for the tip. The configure script doesn't #include <math.h> so
    isinf() is not mapped to __builtin_isinf(). Consequently the conftest link
    fails:

    configure:21401: checking for isinf
    configure:21457: gcc -o conftest -g -O2   conftest.c -lresolv -lsocket -lnsl
    -lrt -ldl  -lm >&5
    conftest.c:252: warning: conflicting types for built-in function 'isinf'
    Undefined                        first referenced
     symbol                                                 in file
    isinf                               /var/tmp//ccmTAet6.o
    ld: fatal: Symbol referencing errors. No output written to conftest
    collect2: ld returned 1 exit status
    

    I found this code in /usr/include/iso/math_c99.h, included by <math.h>:

        #undef      isinf
        #define     isinf(x)        __builtin_isinf(x)

    Skip

    @mdickinson
    Copy link
    Member

    The segfault is a little worrying; I don't understand that at all.

    Skip, can you come up with a configure patch that would allow isinf to be
    detected on Solaris?

    I'll also patch Py_IS_INFINITY to make sure that it forces its argument
    into memory before testing it; this *should* fix the problem on platforms
    that don't have isinf. (Note to self: first find out whether
    Py_IS_INFINITY is ever applied to single precision floats, or whether we
    can assume the argument is always a double.)

    @smontanaro
    Copy link
    Contributor Author

    can you come up with a configure patch that would allow isinf to be
    detected on Solaris?

    The plot thickens. I know squat about autoconf sorts of things so I
    asked on the autoconf mailing list. Eric Drake responded (in part):

    The Python ACHECK_FUNCS test should be rewritten (with proper m4
    quoting) as:

      AC_CHECK_FUNCS([acosh asinh atanh expm1 finite log1p])
      AC_CHECK_DECLS([isinf, isnan], [], [], [[#include <math.h>]])

    so I gave that a whirl. Whaddya know? isinf *really* isn't available,
    at least not until C99 apparently. The AC_CHECK_DECLS macro generates
    a conftest.c which looks like this:

      #include <math.h> 
     
      int 
      main () 
      { 
      #ifndef isinf 
        (void) isinf; 
      #endif 
     
        ; 
        return 0; 
      } 

    which fails to compile/link on our Sol10 boxes. Turns out libm.so
    doesn't export an _isinf symbol. The macro I found (in iso/math_c99.h)
    doesn't expand in gcc -E output of the above code, so in my non-c99
    environment I don't get to use isinf. Looks like your Py_IS_INFINITY
    fix will be necessary. Eric also suggested using the gnu isinf and
    isnan modules:

    http://git.savannah.gnu.org/gitweb/?
    p=gnulib.git;a=blob;f=m4/isinf.m4;h=1b9e45a;hb=67461c3

    I'll leave that for you to decide.

    In any case, the above changes to AC_CHECK_FUNCS should probably be
    made.

    Skip

    @mdickinson
    Copy link
    Member

    Thanks, Skip. It looks like the top priority is fixing Py_IS_INFINITY,
    then (still assuming that I'm not barking up entirely the wrong tree).
    I've opened bpo-4575 for the Py_IS_INFINITY fix.

    I'll look at the changes to AC_CHECK_FUNCS, too.

    @mdickinson
    Copy link
    Member

    autoconf checks for isinf and isnan fixed in r68299.
    I also added a check for isfinite, which should really be used in
    preference to finite: isfinite is standard in C99, while finite doesn't
    seem to be part of any standard.

    @bitdancer
    Copy link
    Member

    NB: I fixed the test_posix failure on trunk/2.6/py3k/3.1 in r73908,
    r73914, r73913, and r73915.

    @bitdancer
    Copy link
    Member

    Are there any open problems left here or can this bug be closed?

    @terryjreedy
    Copy link
    Member

    This appears to be fixed.

    Skip: keywords now has a '-no selection-' option to get rid of keywords

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    None yet
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants