classification
Title: Decimal to receive from_float method
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: facundobatista, mark.dickinson, rhettinger, steven.daprano
Priority: normal Keywords: needs review

Created on 2008-12-31 22:59 by steven.daprano, last changed 2009-01-11 05:03 by rhettinger. This issue is now closed.

Files
File name Uploaded Description Edit
decimal2.diff rhettinger, 2009-01-03 09:38 Add from_float() methods
Messages (20)
msg78664 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2008-12-31 22:59
In the PEP for Decimal, it was discussed that the class should have a 
from_float() method for converting from floats, but to leave it out of 
the Python 2.4 version:
http://www.python.org/dev/peps/pep-0327/#from-float

Following discussions with Mark Dickinson, I would like to request 
that this now be implemented.

The suggested API is:
Decimal.from_float(floatNumber, [decimal_places])

At the risk of derailing the request, I wonder whether it is better to 
give a context rather than a number of decimal places?

Pro: better control over conversion, as you can specify rounding 
method as well as number of decimal places.

Con: more difficult for newbies to understand.

Semi-pro: float to decimal conversions are inherently tricky, perhaps 
we should be discouraging newbies from blindly calling from_float() 
and make the (hypothetical) context argument mandatory rather than 
optional?
msg78670 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-01 00:38
The decimal constructor should be lossless.  The underlying spec was
designed with the notion that all numbers in decimal are exact;
operations can be lossy but the numbers themselves are exact. 
Accordingly, I recommend Decimal.from_float(f) with no qualifiers or
optional arguments.

To support the use case of wanting to round the input, I suggest a
separate method modeled on Context.create_decimal().  It can either be
an extension of the existing method or a new method like
Context.create_decimal_from_float().  I recommend the former since
rounding is already implied by the context qualifier.  Either way, the
effect would be the same as Decimal.from_float(f) + 0 in a given
context.  Per the docs:  "Creates a new Decimal instance from num but
using self as context. Unlike the Decimal constructor, the context
precision, rounding method, flags, and traps are applied to the conversion."
msg78671 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-01 00:43
FYI, there is already a lossless implementation in the docs:

def float_to_decimal(f):
    "Convert a floating point number to a Decimal with no loss of
information"
    n, d = f.as_integer_ratio()
    with localcontext() as ctx:
        ctx.traps[Inexact] = True
        while True:
            try:
               return Decimal(n) / Decimal(d)
            except Inexact:
                ctx.prec += 1
msg78781 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-02 11:31
> Accordingly, I recommend Decimal.from_float(f) with no qualifiers or
> optional arguments.

I agree with this.  Since lossless conversion is feasible, this seems 
the obvious way to go.

It's lucky that the exponent range for (binary) floats is relatively 
small, though:  for IEEE 754 floats the worst case conversions (e.g., 
(2**53-1)/2**1074) produce Decimals with something like 767 significant 
digits, which is still plenty small enough not to cause difficulties.

The recipe in the docs is not industrial strength: it doesn't handle 
infinities, nans and negative zero.  I think there's a place for a 
from_float method that handles these special cases correctly---or at 
least as correctly as reasonable:  +-infinity should convert to +-
infinity, nans to (quiet) nans.  I don't think it's worth worrying about 
preserving the sign or payload of a binary nan: just convert all float 
nans to Decimal('NaN').    I do however think it's worth trying to 
preserve the sign of negative zero.  Note: I'm not suggesting changing 
the *documentation* at all---the recipe there is fine as it is, IMO, but 
I think an official version should include these corner cases.
msg78784 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-02 11:48
One also has to worry about the exponent of the converted result:  e.g., 
should Decimal.from_float(10.0) produce Decimal('1E1') or Decimal('10')?
The latter looks nicer, to me.

IEEE 754 isn't much help here:  as far as I can tell it says nothing about 
binary <-> decimal conversions.

I see two reasonable strategies:  (1) always use the largest exponent 
possible (so we'd get Decimal('1E1') above), or (2) when the quantity 
converted is an exact integer, use an exponent of zero; otherwise fall 
back to (1).

Option (2) is pretty much what the recipe in the docs does already, I 
think:  it computes a quotient of two Decimals, each having exponent zero, 
so the preferred exponent of the result is also zero.
msg78789 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2009-01-02 13:05
Mark suggested the following strategy for Decimal.from_float: "always 
use the largest exponent possible".

Just for the avoidance of all doubt, do you mean the largest exponent 
with the number normalised to one digit to the right of the decimal 
place? Because 1e1 = 0.1e2 = 0.01e3 = ... and there is no "largest 
exponent possible" if you allow unnormalised numbers. Seems obvious to 
me, but maybe I'm missing something.
msg78793 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2009-01-02 13:18
Raymond:
> Accordingly, I recommend Decimal.from_float(f) with no 
> qualifiers or optional arguments.

-0 on this one. It's going to confuse an awful lot of newbies when 
they write Decimal.from_float(1.1) and get 
Decimal('110000000000000008881784197001252...e-51').

Also, why not just extend the Decimal() constructor to accept a float 
as the argument? Why have a separate from_float() method at all?

> To support the use case of wanting to round the input, I 
> suggest a separate method modeled on Context.create_decimal().

+1 on this.
msg78796 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-02 14:09
> Just for the avoidance of all doubt, do you mean the largest exponent 
> with the number normalised to one digit to the right of the decimal 
> place?

No.  I'm using 'exponent' in the sense described in the standard.  See:

http://speleotrove.com/decimal/dbover.html

Equivalently, it's the value of the _exp attribute for a Decimal 
instance.  (For the purposes of disambiguation, the alternative exponent 
that you describe above is often referred to as the 'adjusted exponent' 
in the documentation and code.)

Briefly, every finite Decimal can be thought of as a triple (sign, 
coefficient, exponent), representing the value (-1)**sign * coefficient 
* 10**exponent, with the coefficient an integer.  It's this exponent 
that should be maximized.

> Because 1e1 = 0.1e2 = 0.01e3 = ... and there is no "largest 
> exponent possible"

All these have exponent 1:

>>> Decimal('1e1')._exp
1
>>> Decimal('0.1e2')._exp
1
>>> Decimal('0.01e3')._exp
1

IOW, leading zeros have no significance;  only trailing zeros do.

> Also, why not just extend the Decimal() constructor to accept a float 
> as the argument? Why have a separate from_float() method at all?

This was discussed extensively when the decimal module was being 
proposed;  see the Decimal PEP for arguments against this.
msg78843 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2009-01-02 17:08
Mark wrote:
>> Also, why not just extend the Decimal() constructor to accept
>> a float as the argument? Why have a separate from_float() 
>> method at all?
>
> This was discussed extensively when the decimal module was 
> being proposed;  see the Decimal PEP for arguments against this.

I'm very aware of that. The Decimal PEP says the consensus was for 
from_float() to take a second argument specifying the precision. 
Decimal(1.1) => Decimal("1.1") was rejected for the reasons given in 
the PEP by Paul Moore, and Decimal(1.1) => 
Decimal('110000000000000008881784197001252...e-51') was (presumably) 
rejected because it would confuse newbies. Hence the decision to (1) 
make an alternative constructor and (2) have it take a second 
argument.

It looks like you and Raymond have rejected #2 but are keeping #1, and 
I'm curious why. That's genuine curiosity, and a real question, not a 
thinly-veiled scowl of disapproval disguised as a question :)

Anyway, I'm happy enough so long as Raymond's suggested 
Context.create_decimal() exists, that's the actual functionality I'm 
after, so maybe I should let you guys get on with it. Thanks.
msg78846 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-02 17:28
> It looks like you and Raymond have rejected #2 but are keeping #1

I'm not against #2, but I'm not particularly for it either.  In any case, 
once you've converted your float to Decimal it's trivial to round it to 
whatever precision you feel like, so #2 seems unnecessary to me.  -0.0.

I am -1.100000000000000088817841970012523233890533447265625 on any 
implementation of from_float (with or without keywords, defaults, etc.) 
for which Decimal.from_float(1.1) gives Decimal('1.1').
msg78875 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-02 20:44
> once you've converted your float to Decimal it's trivial 
> to round it to whatever precision you feel like, so #2 
> seems unnecessary to me.

Agree with Mark on how to control rounding.  The DecimalWay(tm) is that
used by Context.create_decimal().  A second argument is problematic for
two reasons.  First, it demands that some rounding take place but
doesn't let you control the rounding method or signal an Inexact
conversion -- you need a context for that.  Second, the API for decimal
is already somewhat complex and we don't want to make it worse by
introducing a new variant with a second argument that has no parallel
elsewhere in the API.

Also agree with Mark regarding Decimal.from_float() which needs to be
lossless and exact.  It parallels fractions.from_float() which does the
same thing for rationals.

The DecimalWay(tm) is to have Decimal constructors be exact and to use
Context based constructors when rounding is desired.

I'll submit a patch to this effect (unless someone beats me to it).
msg78931 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-03 04:33
See attached patch for Py27.
msg78942 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-03 10:40
Instead of the repeated divisions and Inexact tests, how about a direct 
approach:  n/2**k = (n*5**k)/10**k, so something like:

    sign = 0 if copysign(1.0, self) == 1.0 else 1
    n, d = abs(self).as_integer_ratio()
    k = d.bit_length() - 1
    return _dec_from_triple(sign, str(n*5**k), -k)

should work, and would likely be faster too.

I also think the sign of 0 should be preserved:  i.e.,

>>> Decimal.from_float(-0.0)
Decimal('-0')

Am still reviewing---more comments to come.
msg78946 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-03 11:19
A couple more things:

1. There's a typo 'equilvalent' in the decimal.py part of the patch.

2. Can I suggest using

return d._fix(self)

instead of 

return self.plus(d)

in create_decimal_from_float.  The plus method does two things: rounds 
to the current context *and* converts -0.0 to 0.0; we only want the 
first of these.

(It's always been a source of mild irritation to me that there's no 
public method for simply rounding a Decimal to a given context---i.e., a 
public equivalent of _fix.)
msg78993 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-03 19:21
The direct method is *much* faster!
Applied Mark's suggestions.
Committed as r68208 and r68211 .
msg79045 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-04 11:31
Raymond,

Do you think it would be worth replacing the two uses of
conditional expressions in Decimal.from_float with if-else
statements?

Alex Goretoy pointed out (on c.l.p) that the trunk version of
decimal.py can no longer be imported into Python 2.4 (and 2.3).
I don't know how much this matters, but it seems to go against
the comments about 2.3 compatibility at the top of decimal.py.
I admit that I don't really understand the motivation for these
comments, or whether they're still relevant 4 versions on
from Python 2.3.

Of course, from_float still won't work with earlier versions
of Python, but having one Decimal method unavailable seems
like a lesser crime than making 'import decimal' fail.
msg79084 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-04 20:20
> Do you think it would be worth replacing the two uses of
> conditional expressions in Decimal.from_float with if-else
> statements?

Yes, please.

> Of course, from_float still won't work with earlier versions
> of Python, but having one Decimal method unavailable seems
> like a lesser crime than making 'import decimal' fail.

Right.  I don't see an easy way around that short of having
a conditional compilation, allowing use of alternative slow
code multiplying the float repeatedly by two to build-up
the float digits.
msg79579 - (view) Author: Facundo Batista (facundobatista) * (Python committer) Date: 2009-01-10 21:43
Raymond, Mark, thanks for this work!

I'd include the following in the PEP (under the "from float"
discussion), what do you think?

"""
Update: The .from_float() method was added to Python 2.7 and 3.1
versions, providing lossless and exact conversion from float to Decimal 
(see issue 4796 [7]_ for further information).


It has the following syntax::

    Decimal.from_float(floatNumber)

where ``floatNumber`` is the float number origin of the construction.  
Example::

    >>> Decimal.from_float(1.1)
    Decimal('1.100000000000000088817841970012523233890533447265625')
"""
msg79580 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2009-01-10 22:00
I agree the PEP should be updated.  Your proposed update looks good to me.
msg79588 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-01-11 05:03
Go ahead with the update, but it isn't really necessary.  The PEPs are
for getting something into the language in the first place and for
centralizing a major discussion.  They typically get out of date quickly
after they've been implemented.  In this case, we've got the tracker
discussion to document the minor discussions than led to this minor
change in the API.  So, it's okay to update the PEP or not.
History
Date User Action Args
2009-01-11 05:03:48rhettingersetmessages: + msg79588
2009-01-10 22:00:36mark.dickinsonsetmessages: + msg79580
2009-01-10 21:43:11facundobatistasetnosy: + facundobatista
messages: + msg79579
2009-01-04 20:20:15rhettingersetmessages: + msg79084
2009-01-04 11:31:56mark.dickinsonsetmessages: + msg79045
2009-01-03 19:21:03rhettingersetstatus: open -> closed
resolution: fixed
messages: + msg78993
versions: + Python 3.1, Python 2.7
2009-01-03 11:19:29mark.dickinsonsetassignee: mark.dickinson -> rhettinger
messages: + msg78946
2009-01-03 10:40:42mark.dickinsonsetmessages: + msg78942
2009-01-03 09:38:47rhettingersetfiles: - decimal.diff
2009-01-03 09:38:40rhettingersetkeywords: + needs review, - patch
assignee: rhettinger -> mark.dickinson
files: + decimal2.diff
2009-01-03 04:33:24rhettingersetfiles: + decimal.diff
keywords: + patch
messages: + msg78931
2009-01-02 20:44:14rhettingersetmessages: + msg78875
2009-01-02 17:28:26mark.dickinsonsetmessages: + msg78846
2009-01-02 17:08:25steven.dapranosetmessages: + msg78843
2009-01-02 14:09:35mark.dickinsonsetmessages: + msg78796
2009-01-02 13:18:08steven.dapranosetmessages: + msg78793
2009-01-02 13:05:35steven.dapranosetmessages: + msg78789
2009-01-02 11:48:37mark.dickinsonsetmessages: + msg78784
2009-01-02 11:31:38mark.dickinsonsetnosy: + mark.dickinson
messages: + msg78781
2009-01-01 00:43:08rhettingersetmessages: + msg78671
2009-01-01 00:38:23rhettingersetmessages: + msg78670
2008-12-31 23:06:05rhettingersetassignee: rhettinger
nosy: + rhettinger
2008-12-31 22:59:19steven.dapranocreate