classification
Title: geometric_mean can raise OverflowError for large input length
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: steven.daprano Nosy List: mark.dickinson, steven.daprano, vstinner
Priority: normal Keywords:

Created on 2016-09-13 02:20 by steven.daprano, last changed 2016-10-04 16:32 by steven.daprano.

Files
File name Uploaded Description Edit
geometric_mean_long_input.py mark.dickinson, 2016-10-03 19:18 review
geometric_mean_long_input_v2.py mark.dickinson, 2016-10-03 20:08 review
Messages (6)
msg276149 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2016-09-13 02:20
>>> statistics.geometric_mean([0.7 for _ in range(5000)])
Traceback (most recent call last):
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 362, in float_nroot
    isinfinity = math.isinf(x)
OverflowError: int too large to convert to float

See #27975 and #27181
msg276201 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-09-13 07:40
For reference, here's the full traceback. Issue #27975 isn't relevant here - the problem is an out-of-range integer being passed to the math.pow operation.

taniyama:cpython-git mdickinson$ ./python.exe
Python 3.7.0a0 (default, Sep 13 2016, 08:36:28) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import statistics
>>> statistics.geometric_mean(0.7 for _ in range(5000))
Traceback (most recent call last):
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 343, in float_nroot
    isinfinity = math.isinf(x)
OverflowError: int too large to convert to float

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 578, in geometric_mean
    s = 2**p * _nth_root(2**q, n)
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 330, in nth_root
    return _nroot_NS.float_nroot(x, n)
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 345, in float_nroot
    return _nroot_NS.bignum_nroot(x, n)
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 472, in bignum_nroot
    b = 2**q * _nroot_NS.nroot(2**r, n)
  File "/Users/mdickinson/Python/cpython-git/Lib/statistics.py", line 365, in nroot
    r1 = math.pow(x, 1.0/n)
OverflowError: int too large to convert to float
msg276733 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-09-16 18:24
See msg276732 in issue 27761 for a possible solution.
msg277992 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-10-03 19:17
The attached patch provides a fix, and is based partly on Tim Peters' suggestions and code from the issue 27761 discussion. It needs tests. For convenience, it includes the fix for #28327, since it needs the `_frexp_gen` function added there.
msg277993 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-10-03 19:18
And here's the actual patch. :-)
msg277999 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-10-03 20:08
Whoops; that patch was incomplete (it was missing the change to the geometric_mean function itself). Here's an updated patch.
History
Date User Action Args
2016-10-04 16:32:34steven.dapranosetversions: - Python 3.6
2016-10-03 20:08:13mark.dickinsonsetfiles: + geometric_mean_long_input_v2.py

messages: + msg277999
2016-10-03 19:18:42mark.dickinsonsetfiles: + geometric_mean_long_input.py

messages: + msg277993
2016-10-03 19:17:53mark.dickinsonsetversions: + Python 3.7
2016-10-03 19:17:41mark.dickinsonsetmessages: + msg277992
2016-09-16 19:09:17mark.dickinsonsettitle: geometric_mean can raise OverflowError when checking for inf -> geometric_mean can raise OverflowError for large input length
2016-09-16 18:24:13mark.dickinsonsetmessages: + msg276733
2016-09-13 07:40:17mark.dickinsonsetmessages: + msg276201
2016-09-13 02:20:33steven.dapranocreate