This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: math.log and math.log10 domain error on very large Fractions
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Camion, gvanrossum, mark.dickinson, rhettinger, steven.daprano, tim.peters
Priority: normal Keywords:

Created on 2021-01-10 22:44 by Camion, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (12)
msg384783 - (view) Author: (Camion) Date: 2021-01-10 22:44
Python is able to computer the logarithms of very large integers and fractions (by using log(num)-log(denom)), However the fractions.Fraction class fails to do it and raises a math domain error exception.

>>> import math, fractions
>>> f=fractions.Fraction(math.factorial(10000), math.factorial(20000)+1)
>>> math.log(f.numerator)-math.log(f.denominator)
>>> math.log(f)
Traceback (most recent call last):
  File "<pyshell#30>", line 1, in <module>
ValueError: math domain error
>>> math.log10(f.numerator)-math.log10(f.denominator)
>>> math.log10(f)
Traceback (most recent call last):
  File "<pyshell#32>", line 1, in <module>
ValueError: math domain error
msg384784 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-01-10 23:09
Internally, the math.log10() function only works on floats.  By way of the __float__() method, the input to the the math.log10() function is asked to convert itself to a float.  Fractions implement __float__() by just dividing the numerator and denominator.  In your example, the result is smaller than the smallest representable positive float (2**-1074), so the result gets rounded down to 0.0.  The log10() function isn't defined for zero:

    >>> f = Fraction(factorial(10000), factorial(20000)+1)
    >>> float(f)
    >>> log10(f)
    Traceback (most recent call last):
    ValueError: math domain error

Short of adding a log10() method to the Fraction class, there isn't a straight forward way to accommodate getting this to work.
msg384786 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-10 23:47
I guess another approach might be to change the math module so that all its functions support franctions (and decimal, and other esoteric number types). But that would require having __log__ and many others as methods on the numeric types.

Personally I wish we had kept the math module for C doubles (Python floats) only -- I fear that it has become a mash-up of such functions with random other things people may remember from their high school math classes such as factorial(). This is just asking for misunderstandings like this. :-(
msg384787 - (view) Author: (Camion) Date: 2021-01-10 23:51
math.log10 works perfectly on integers which are too large to be converted to floats. I see no reason why it couldn't work as well with fractions.

>>> math.log10(math.factorial(10000))
>>> math.log10(math.factorial(1000000))
>>> math.log10(float(math.factorial(10000)))
Traceback (most recent call last):
  File "<pyshell#36>", line 1, in <module>
OverflowError: int too large to convert to float
msg384788 - (view) Author: (Camion) Date: 2021-01-10 23:58
A generic solution might be to consider all number as fractions (which they are) (floats have denominators being 2^n, decimals have denominators being 10^n, integers have denominators being 1, and fractions have denominators being integers... but this would be a heavy change.
msg384791 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-01-11 00:50
> math.log10 works perfectly on integers which are too large
> to be converted to floats. I see no reason why it couldn't 
> work as well with fractions.

This is tricky territory.  Mathematically, a common logarithm can and often will have a irrational result, so floats will enter the picture at some point.  Unlike Fractions which don't have size limits, floats are range limited, so some overflows and underflows will still happen regardless. 

> ... but this would be a heavy change.

There's the rub.  We could create a dunder method for every math function and require that every numeric type to support every dunder method.

That's a somewhat major design change and it's far above my pay grade to say whether that would be worth it ;-)
msg384792 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2021-01-11 02:32
Any principled change costs more than it's worth :-(

I'm mostly sympathetic with Guido's view, and have long advocated a new `imath` module to hold the ever-growing number of functions that are really part of integer combinatorics.  But it's become nearly impossible to add a new module anymore:  python-ideas can't maintain any focus, python-dev has become an eternal circular debating society, and "put it on pyPI first to see whether it's adopted" has become a go-to bait-&-switch scam (e.g., witness Grant Jenks's `sortedcontainers` module, which is absolutely first rate, and widely used, but "why add it to the core? it's already on pyPI" has shut down any suggestion to fold it in - although I don't know that Grant would _want_ to fold it in).

OTOH, I don't agree `math` should be limited solely to floats.  It would be, e.g., pointlessly annoying for things like `pow(x, 4)` or `atan(1)` to blow up just because an argument is an int instead of a float.  I have no problem with limiting `math` functions to the simple scalar types that also work in C libm calls (where ints aren't a problem because the compiler inserts coercions to double). As a slight extension then, I have no problem either with log(1 << 1000000) working, because Python doesn't have C's distinctions between integer sizes.  So, overall, "floats or ints, but generally just the functions C's <math.h> defines".  Which is pretty much how `math` started.

But I don't see a realistic way to get there from here. Even though I'm retired now, I don't _want_ to spend 16 hours arguing every day for months - that's a major percentage of my remaining expected life ;-)

For other approaches, people doing serious work already solve it the obvious way. For example, the spiffy arbitrary-precision `mpmath` binary float module supplies its own versions of libm functions, that work on the numeric types it defines. It's not a real problem in real life that they don't also work on Fraction or Decimal. In fact, I'm glad they don't! It would usually be "an error" in my code if I tried to pass a Decimal to an mpmath function, because I'm using mpmath to begin with precisely because I _intend_ to work specifically and only with mpmath's numeric types for the duration.

So we can keep this report open forever waiting for a miracle, or yield to reality and close it as "sorry - won't fix" now ;-)
msg384793 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-01-11 03:04
> Any principled change costs more than it's worth :-(

For anyone wanting to go forward with this, here's a summary of the feature request:

    Given two non-zero same signed integers x and y,
    log10(Fraction(x, y)) does the same as log10(x / y).

    This works well unless the ratio of x to y is outside
    the range supported by floats.

    What is desired is that for extreme values of x and y,
    that log10() will treat the fraction differently and
    compute log10(x) - log10(y) instead of log10(x / y).
msg384794 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-11 03:33
The problem with that formulation of the feature request is that it would require the math.log function to recognize rationals, or at least the Fraction type. Since the numeric tower and the fractions module are implemented in pure Python that would be somewhat problematic.

I think people who work a lot with fractions and want to compute their logarithm just need to use the formula that the OP gave.
msg384803 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-01-11 08:22
@Camion: Can you say more about your use-case? One thing that's on my would-be-nice-to-do-one-day list is 'e', 'f' and 'g'-style decimal formatting for Fraction objects. Without that, it's often hard to get a quick sense of the magnitude of a Fraction without conversion to float. Would that formatting capability help at all with your use-case, or are you doing something entirely different?
msg384806 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2021-01-11 09:49
Guido wrote:

> it would require the math.log function to recognize rationals

I don't think it would. You don't need a type-check, you just need to check for a pair of numerator and denominator attributes.

So we could do something like this, except in C:

        return log(float(x))
    except DomainError:
            a = x.numerator
            b = x.denominator
        except AttributeError:
            return log(a) - log(b)
msg384859 - (view) Author: (Camion) Date: 2021-01-11 19:49
@mark.dickinson, I fell on this problem in the reached precision evaluation, of a multiprecision algorithm designed to compute decimals of PI (using "John Machin like" formulas). 

The fact is that fractions module is really nice to implement this kind of multiprecision algorithm and math.log10 on each subsequent term of the series gives you an immediate evaluation of the current reached precision.

I agree that the solution I used work pretty well, but it makes me sad because it's far from beautifull.

I think one should consider that there are two cases with different objectives : floats and the likes which target speed, and those multiprecision types which target - well... precision. I wonder if the solution would not be to implement those functions in the fractions and decimal libraries (just calling log on integer for numerator and denominator).
Date User Action Args
2022-04-11 14:59:40adminsetgithub: 87052
2021-01-11 19:49:16Camionsetmessages: + msg384859
2021-01-11 09:49:08steven.dapranosetnosy: + steven.daprano
messages: + msg384806
2021-01-11 08:22:38mark.dickinsonsetnosy: + mark.dickinson
messages: + msg384803
2021-01-11 03:33:35gvanrossumsetstatus: open -> closed
resolution: wont fix
messages: + msg384794

stage: resolved
2021-01-11 03:04:28rhettingersetmessages: + msg384793
2021-01-11 02:32:56tim.peterssetnosy: + tim.peters
messages: + msg384792
2021-01-11 00:50:46rhettingersetmessages: + msg384791
2021-01-10 23:58:50Camionsetmessages: + msg384788
2021-01-10 23:51:43Camionsetmessages: + msg384787
2021-01-10 23:47:32gvanrossumsetnosy: + gvanrossum
messages: + msg384786
2021-01-10 23:09:41rhettingersetnosy: + rhettinger
messages: + msg384784
2021-01-10 22:44:43Camioncreate