This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Decimal handling error in statistics module
Type: behavior Stage:
Components: Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: statistics._decimal_to_ratio() produces non-integer ratio
View: 20536
Assigned To: Nosy List: ncoghlan, oscarbenjamin, python-dev, skrah, steven.daprano, wolma
Priority: normal Keywords:

Created on 2014-02-08 11:10 by wolma, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (4)
msg210613 - (view) Author: Wolfgang Maier (wolma) * Date: 2014-02-08 11:10
Can this still be fixed in 3.4 ??

I came across this bug in the statistics module today:

>>> import statistics
>>> data = [Decimal('1e4')]
>>> statistics.mean(data)
Traceback (most recent call last):
  File "<pyshell#465>", line 1, in <module>
    statistics.mean(data)
  File "C:/Python33\statistics.py", line 316, in mean
    return _sum(data)/n
  File "C:/Python33\statistics.py", line 167, in _sum
    total += Fraction(n, d)
  File "C:\Python33\lib\fractions.py", line 163, in __new__
    raise TypeError("both arguments should be "
TypeError: both arguments should be Rational instances

The reason for this is that the statistics module converts Decimals to exact ratios internally like this:

def _decimal_to_ratio(d):
    """Convert Decimal d to exact integer ratio (numerator, denominator).

    >>> from decimal import Decimal
    >>> _decimal_to_ratio(Decimal("2.6"))
    (26, 10)

    """
    sign, digits, exp = d.as_tuple()
    if exp in ('F', 'n', 'N'):  # INF, NAN, sNAN
        assert not d.is_finite()
        raise ValueError
    num = 0
    for digit in digits:
        num = num*10 + digit
    if sign:
        num = -num
    den = 10**-exp
    #if type(den) != int:
    #    print (d, sign, digits, exp, num, den)
    return (num, den)
	
The important part here is the line:

den = 10**-exp

which makes den an int if - and **only** if - exp is negative.

However, the exponent in the namedtuple returned by Decimal.as_tuple() is not guaranteed to be negative and in fact:

>>>Decimal('1e4').as_tuple()
DecimalTuple(sign=0, digits=(1,), exponent=4)

With this den in the above function becomes a float, which raises an error subsequently in _sum, when it essentially tries to do:
Fraction(num, den)

IMPORTANT: I have been running this example on my laptop with Python3.3.0 and the statistics module imported from sys.path.
Thus, I cannot exclude that the decimal module has changed since 3.3.0 and is now always returning a negative exponent with
.as_tuple(). I'm abroad and cannot verify this against the Python3.4dev build.
Could somebody confirm this for 3.4 and reject this issue immediately if the error doesn't occur ?

If this is not version dependent though, then there is a very simple bug-fix for it, which I suggest should still be incorporated into 3.4:

if exp >= 0:
    return int(x), 1

inserted right after the raise ValueError line is doing the job.
I cannot generate the diff for this myself, so could somebody do this.

Thanks a lot for your efforts!
msg210615 - (view) Author: Wolfgang Maier (wolma) * Date: 2014-02-08 11:12
#if type(den) != int:
    #    print (d, sign, digits, exp, num, den)

was inserted by me of course for debugging this. Forgot to take it out again.
msg210617 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-02-08 11:29
This looks like a duplicate of #20536.  Steven, do you think you
have a chance to fix this before rc1?
msg210624 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2014-02-08 13:02
On Sat, Feb 08, 2014 at 11:29:29AM +0000, Stefan Krah wrote:
> This looks like a duplicate of #20536.  Steven, do you think you
> have a chance to fix this before rc1?

Working on it now. Should have a patch and regression tests in 15 
minutes, I'll post it on 20536.
History
Date User Action Args
2022-04-11 14:57:58adminsetgithub: 64760
2014-02-08 13:56:02ncoghlansetstatus: open -> closed
superseder: statistics._decimal_to_ratio() produces non-integer ratio
resolution: duplicate
2014-02-08 13:02:25steven.dapranosetmessages: + msg210624
2014-02-08 11:35:34skrahsetkeywords: - 3.4regression
2014-02-08 11:29:29skrahsetkeywords: + 3.4regression

messages: + msg210617
2014-02-08 11:18:43wolmasetnosy: + ncoghlan
2014-02-08 11:12:26wolmasetmessages: + msg210615
2014-02-08 11:10:27wolmacreate