classification
Title: Use C99 functions in math if available
Type: enhancement Stage: resolved
Components: Extension Modules Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, mark.dickinson, serhiy.storchaka, zach.ware
Priority: normal Keywords: patch

Created on 2016-01-15 11:42 by serhiy.storchaka, last changed 2017-09-15 16:57 by paulromano. This issue is now closed.

Files
File name Uploaded Description Edit
math_libc_funcs.patch serhiy.storchaka, 2016-01-15 11:42 review
Pull Requests
URL Status Linked Edit
PR 515 merged serhiy.storchaka, 2017-03-06 13:11
PR 632 merged serhiy.storchaka, 2017-03-12 08:39
PR 637 merged serhiy.storchaka, 2017-03-12 10:48
PR 703 larry, 2017-03-17 21:00
PR 3603 merged paulromano, 2017-09-15 16:57
Messages (15)
msg258288 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-15 11:42
Currently the math module uses own implementation of some mathematical functions that are in C99 standard, but not in C89 standard: tgamma, lgamma, erf, erfc. Proposed patch makes it to use functions from standard C library if they are available. They are faster and presumably more accurate.

Here are microbenchmark results (time in microseconds):

                  0.1     1      3     10     30
erf unpatched:   0.506  0.655  0.509  0.548  0.239
erf patched:     0.129  0.252  0.357  0.253  0.253

erfc unpatched:  0.508  0.646  0.532  0.522  0.251
erfc patched:    0.129  0.239  0.373  0.371  0.307

                    0.1    1.5     3     10     10.5
gamma unpatched:   0.369  0.279  0.273  0.274  0.457
gamma patched:     0.24   0.23   0.412  0.741  0.682

lgamma unpatched:  0.351  0.338  0.478  0.627  0.52
lgamma patched:    0.217  0.155  0.37   0.372  0.247

If some libm implementations are pretty bad, they can be disabled by undefining corresponding HAVE_XXX macros.
msg258293 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-01-15 13:28
Faster: maybe. More accurate ... Mark can tell you some funny stories. :)
msg258505 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-01-18 08:55
LGTM. What platform are the benchmarks on?

Is there some way that this can be submitted to the buildbots *before* applying to the 3.6 branch, so that we can get a sense of how widespread (if at all) libm problems are?
msg258508 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-18 09:04
The benchmarks were ran on 32-bit Linux with AMD processor.

I'll try to run custom build on buildbots.
msg289099 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-06 13:12
I don't know how to run custom build on buildbots with new workflow.
msg289457 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2017-03-11 20:47
So the GitHub branch fails on my OS X 10.10 machine: there are a number of test failures for gamma and lgamma (but none for erf and erfc).

Some of the gamma and lgamma test failures are accuracy issues; some are more likely to do with errno handling: OS X probably isn't setting errno appropriate on domain or range errors, so any implementation using the libm lgamma/tgamma would need to add code for error handling.

Given this, I'd prefer to leave lgamma and gamma as-is, using their current implementations, at least on OS X; I don't regard the accuracy loss in using the libm functions to be within acceptable limits.

There may still be a case for using the libm versions of erf and erfc.
msg289478 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-12 05:40
Tests are failed only on s390x RHEL 3.x.

http://buildbot.python.org/all/builders/s390x%20RHEL%203.x/builds/446/steps/test/logs/stdio
======================================================================
FAIL: test_mtestfile (test.test_math.MathTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z/build/Lib/test/test_math.py", line 1287, in test_mtestfile
    '\n  '.join(failures))
AssertionError: Failures in test_mtestfile:
  gam0003: gamma(-inf): expected 'ValueError', got nan (not equal)
  gam0045: gamma(1e-160): expected 1e+160, got 1.0000000000000063e+160 (error = 6.24e+145 (40 ulps); permitted error = 0 or 20 ulps)
  gam0046: gamma(1e-308): expected 1e+308, got 1.0000000000000136e+308 (error = 1.36e+294 (68 ulps); permitted error = 0 or 20 ulps)
  gam0047: gamma(5.6e-309): expected 1.7857142857142848e+308, got 1.785714285714199e+308 (error = 8.58e+294 (430 ulps); permitted error = 0 or 20 ulps)
  gam0065: gamma(-1e-160): expected -1e+160, got -1.0000000000000063e+160 (error = 6.24e+145 (40 ulps); permitted error = 0 or 20 ulps)
  gam0066: gamma(-1e-308): expected -1e+308, got -1.0000000000000136e+308 (error = 1.36e+294 (68 ulps); permitted error = 0 or 20 ulps)
  gam0067: gamma(-5.6e-309): expected -1.7857142857142848e+308, got -1.785714285714199e+308 (error = 8.58e+294 (430 ulps); permitted error = 0 or 20 ulps)
  gam0081: gamma(-1.0000000000000002): expected 4503599627370495.5, got 4503599627370484.5 (error = 11 (22 ulps); permitted error = 0 or 20 ulps)
  gam0084: gamma(-100.00000000000001): expected -7.540083334883109e-145, got -7.540083334882688e-145 (error = 4.21e-158 (296 ulps); permitted error = 0 or 20 ulps)
  gam0085: gamma(-99.99999999999999): expected 7.540083334884096e-145, got 7.540083334884402e-145 (error = 3.06e-158 (215 ulps); permitted error = 0 or 20 ulps)
  gam0100: gamma(170.0): expected 4.269068009004705e+304, got 4.269068009005011e+304 (error = 3.06e+291 (628 ulps); permitted error = 0 or 20 ulps)
  gam0101: gamma(171.0): expected 7.257415615307999e+306, got 7.257415615308882e+306 (error = 8.83e+293 (708 ulps); permitted error = 0 or 20 ulps)
  gam0102: gamma(171.624): expected 1.7942117599248104e+308, got 1.794211759924979e+308 (error = 1.69e+295 (845 ulps); permitted error = 0 or 20 ulps)
  gam0120: gamma(-100.5): expected -3.3536908198076787e-159, got -3.353690819807666e-159 (error = 1.26e-173 (25 ulps); permitted error = 0 or 20 ulps)
  gam0121: gamma(-160.5): expected -5.255546447007829e-286, got -5.255546447008121e-286 (error = 2.92e-299 (313 ulps); permitted error = 0 or 20 ulps)
  gam0122: gamma(-170.5): expected -3.3127395215386074e-308, got -3.312739521538679e-308 (error = 7.16e-322 (145 ulps); permitted error = 0 or 20 ulps)
  gam0140: gamma(-63.349078729022985): expected 4.177797167776188e-88, got 4.177797167776136e-88 (error = 5.19e-102 (93 ulps); permitted error = 0 or 20 ulps)
  gam0141: gamma(-127.45117632943295): expected 1.183111089623681e-214, got 1.183111089623727e-214 (error = 4.6e-228 (223 ulps); permitted error = 0 or 20 ulps)

----------------------------------------------------------------------
msg289486 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-12 08:42
PR 632 switch on using libc implementation on Windows. Will see how AppVeyor tests passed.
msg289487 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-12 09:00
There are accuracy issues with tgamma() and lgamma() on Windows. But less than on OS X.

======================================================================
FAIL: test_mtestfile (test.test_math.MathTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_math.py", line 1287, in test_mtestfile
    '\n  '.join(failures))
AssertionError: Failures in test_mtestfile:
  lgam0085: lgamma(-99.99999999999999): expected -331.85460524980596, got -331.86198661322277 (error = 0.00738 (129854318490 ulps); permitted error = 1e-15 or 5 ulps)
  gam0085: gamma(-99.99999999999999): expected 7.540083334884096e-145, got 7.484632144060724e-145 (error = 5.55e-147 (38979707393612 ulps); permitted error = 0 or 20 ulps)
  gam0124: gamma(-176.5): expected -1.196e-321, got 0.0 (error = 1.2e-321 (243 ulps); permitted error = 0 or 20 ulps)
----------------------------------------------------------------------

Does this mean that we should not use libc implementations of tgamma() and lgamma() on Windows?
msg289491 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2017-03-12 09:50
Please can we revert to using the existing implementations of lgamma and gamma on all platforms? It's clear that there are many issues with OS-provided implementations.
msg289494 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-12 10:50
PR 637 revert to using the own implementations of lgamma and gamma on all platforms.
msg289499 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-12 12:23
Done.
msg290199 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-24 22:22
New changeset 4125e5c60e24ffcff8031817dc60984335917f59 by Serhiy Storchaka in branch 'master':
bpo-26121: Revert to using the own implementations of lgamma and gamma on all platforms. (#637)
https://github.com/python/cpython/commit/4125e5c60e24ffcff8031817dc60984335917f59
msg290201 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-24 22:23
New changeset 4dadcd4ed7824c7904add78577e6a05864cfe493 by Serhiy Storchaka in branch 'master':
bpo-26121: Use C library implementation for math functions erf() and erfc() on Windows. (#632)
https://github.com/python/cpython/commit/4dadcd4ed7824c7904add78577e6a05864cfe493
msg290225 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2017-03-24 22:27
New changeset 97553fdf9daa8231eb05a1ca9933a2b03b0bdad0 by Mark Dickinson (Serhiy Storchaka) in branch 'master':
bpo-26121: Use C library implementation for math functions: (#515)
https://github.com/python/cpython/commit/97553fdf9daa8231eb05a1ca9933a2b03b0bdad0
History
Date User Action Args
2017-09-15 16:57:46paulromanosetpull_requests: + pull_request3594
2017-03-24 22:27:02mark.dickinsonsetmessages: + msg290225
2017-03-24 22:23:16serhiy.storchakasetmessages: + msg290201
2017-03-24 22:22:53serhiy.storchakasetmessages: + msg290199
2017-03-17 21:00:36larrysetpull_requests: + pull_request622
2017-03-12 20:24:29brett.cannonsetnosy: - brett.cannon
2017-03-12 12:23:40serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg289499

stage: patch review -> resolved
2017-03-12 10:50:42serhiy.storchakasetmessages: + msg289494
2017-03-12 10:48:24serhiy.storchakasetpull_requests: + pull_request526
2017-03-12 09:50:58mark.dickinsonsetmessages: + msg289491
2017-03-12 09:00:27serhiy.storchakasetmessages: + msg289487
2017-03-12 08:42:15serhiy.storchakasetmessages: + msg289486
2017-03-12 08:39:46serhiy.storchakasetpull_requests: + pull_request520
2017-03-12 05:40:24serhiy.storchakasetnosy: + zach.ware
messages: + msg289478
2017-03-11 20:47:25mark.dickinsonsetmessages: + msg289457
2017-03-06 13:12:23serhiy.storchakasetmessages: + msg289099
versions: + Python 3.7, - Python 3.6
2017-03-06 13:11:10serhiy.storchakasetpull_requests: + pull_request424
2016-01-18 09:04:39serhiy.storchakasetmessages: + msg258508
2016-01-18 08:55:41mark.dickinsonsetmessages: + msg258505
2016-01-15 17:04:01yselivanovsetnosy: + brett.cannon
2016-01-15 13:28:42christian.heimessetnosy: + christian.heimes
messages: + msg258293
2016-01-15 11:42:15serhiy.storchakacreate