Issue4796

Created on **2008-12-31 22:59** by **steven.daprano**, last changed **2009-01-11 05:03** by **rhettinger**. This issue is now **closed**.

Files | ||||
---|---|---|---|---|

File name | Uploaded | Description | Edit | |

decimal2.diff | rhettinger, 2009-01-03 09:38 | Add from_float() methods |

Messages (20) | |||
---|---|---|---|

msg78664 - (view) | Author: Steven D'Aprano (steven.daprano) * | Date: 2008-12-31 22:59 | |

In the PEP for Decimal, it was discussed that the class should have a from_float() method for converting from floats, but to leave it out of the Python 2.4 version: http://www.python.org/dev/peps/pep-0327/#from-float Following discussions with Mark Dickinson, I would like to request that this now be implemented. The suggested API is: Decimal.from_float(floatNumber, [decimal_places]) At the risk of derailing the request, I wonder whether it is better to give a context rather than a number of decimal places? Pro: better control over conversion, as you can specify rounding method as well as number of decimal places. Con: more difficult for newbies to understand. Semi-pro: float to decimal conversions are inherently tricky, perhaps we should be discouraging newbies from blindly calling from_float() and make the (hypothetical) context argument mandatory rather than optional? |
|||

msg78670 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-01 00:38 | |

The decimal constructor should be lossless. The underlying spec was designed with the notion that all numbers in decimal are exact; operations can be lossy but the numbers themselves are exact. Accordingly, I recommend Decimal.from_float(f) with no qualifiers or optional arguments. To support the use case of wanting to round the input, I suggest a separate method modeled on Context.create_decimal(). It can either be an extension of the existing method or a new method like Context.create_decimal_from_float(). I recommend the former since rounding is already implied by the context qualifier. Either way, the effect would be the same as Decimal.from_float(f) + 0 in a given context. Per the docs: "Creates a new Decimal instance from num but using self as context. Unlike the Decimal constructor, the context precision, rounding method, flags, and traps are applied to the conversion." |
|||

msg78671 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-01 00:43 | |

FYI, there is already a lossless implementation in the docs: def float_to_decimal(f): "Convert a floating point number to a Decimal with no loss of information" n, d = f.as_integer_ratio() with localcontext() as ctx: ctx.traps[Inexact] = True while True: try: return Decimal(n) / Decimal(d) except Inexact: ctx.prec += 1 |
|||

msg78781 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-02 11:31 | |

> Accordingly, I recommend Decimal.from_float(f) with no qualifiers or > optional arguments. I agree with this. Since lossless conversion is feasible, this seems the obvious way to go. It's lucky that the exponent range for (binary) floats is relatively small, though: for IEEE 754 floats the worst case conversions (e.g., (2**53-1)/2**1074) produce Decimals with something like 767 significant digits, which is still plenty small enough not to cause difficulties. The recipe in the docs is not industrial strength: it doesn't handle infinities, nans and negative zero. I think there's a place for a from_float method that handles these special cases correctly---or at least as correctly as reasonable: +-infinity should convert to +- infinity, nans to (quiet) nans. I don't think it's worth worrying about preserving the sign or payload of a binary nan: just convert all float nans to Decimal('NaN'). I do however think it's worth trying to preserve the sign of negative zero. Note: I'm not suggesting changing the *documentation* at all---the recipe there is fine as it is, IMO, but I think an official version should include these corner cases. |
|||

msg78784 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-02 11:48 | |

One also has to worry about the exponent of the converted result: e.g., should Decimal.from_float(10.0) produce Decimal('1E1') or Decimal('10')? The latter looks nicer, to me. IEEE 754 isn't much help here: as far as I can tell it says nothing about binary <-> decimal conversions. I see two reasonable strategies: (1) always use the largest exponent possible (so we'd get Decimal('1E1') above), or (2) when the quantity converted is an exact integer, use an exponent of zero; otherwise fall back to (1). Option (2) is pretty much what the recipe in the docs does already, I think: it computes a quotient of two Decimals, each having exponent zero, so the preferred exponent of the result is also zero. |
|||

msg78789 - (view) | Author: Steven D'Aprano (steven.daprano) * | Date: 2009-01-02 13:05 | |

Mark suggested the following strategy for Decimal.from_float: "always use the largest exponent possible". Just for the avoidance of all doubt, do you mean the largest exponent with the number normalised to one digit to the right of the decimal place? Because 1e1 = 0.1e2 = 0.01e3 = ... and there is no "largest exponent possible" if you allow unnormalised numbers. Seems obvious to me, but maybe I'm missing something. |
|||

msg78793 - (view) | Author: Steven D'Aprano (steven.daprano) * | Date: 2009-01-02 13:18 | |

Raymond: > Accordingly, I recommend Decimal.from_float(f) with no > qualifiers or optional arguments. -0 on this one. It's going to confuse an awful lot of newbies when they write Decimal.from_float(1.1) and get Decimal('110000000000000008881784197001252...e-51'). Also, why not just extend the Decimal() constructor to accept a float as the argument? Why have a separate from_float() method at all? > To support the use case of wanting to round the input, I > suggest a separate method modeled on Context.create_decimal(). +1 on this. |
|||

msg78796 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-02 14:09 | |

> Just for the avoidance of all doubt, do you mean the largest exponent > with the number normalised to one digit to the right of the decimal > place? No. I'm using 'exponent' in the sense described in the standard. See: http://speleotrove.com/decimal/dbover.html Equivalently, it's the value of the _exp attribute for a Decimal instance. (For the purposes of disambiguation, the alternative exponent that you describe above is often referred to as the 'adjusted exponent' in the documentation and code.) Briefly, every finite Decimal can be thought of as a triple (sign, coefficient, exponent), representing the value (-1)**sign * coefficient * 10**exponent, with the coefficient an integer. It's this exponent that should be maximized. > Because 1e1 = 0.1e2 = 0.01e3 = ... and there is no "largest > exponent possible" All these have exponent 1: >>> Decimal('1e1')._exp 1 >>> Decimal('0.1e2')._exp 1 >>> Decimal('0.01e3')._exp 1 IOW, leading zeros have no significance; only trailing zeros do. > Also, why not just extend the Decimal() constructor to accept a float > as the argument? Why have a separate from_float() method at all? This was discussed extensively when the decimal module was being proposed; see the Decimal PEP for arguments against this. |
|||

msg78843 - (view) | Author: Steven D'Aprano (steven.daprano) * | Date: 2009-01-02 17:08 | |

Mark wrote: >> Also, why not just extend the Decimal() constructor to accept >> a float as the argument? Why have a separate from_float() >> method at all? > > This was discussed extensively when the decimal module was > being proposed; see the Decimal PEP for arguments against this. I'm very aware of that. The Decimal PEP says the consensus was for from_float() to take a second argument specifying the precision. Decimal(1.1) => Decimal("1.1") was rejected for the reasons given in the PEP by Paul Moore, and Decimal(1.1) => Decimal('110000000000000008881784197001252...e-51') was (presumably) rejected because it would confuse newbies. Hence the decision to (1) make an alternative constructor and (2) have it take a second argument. It looks like you and Raymond have rejected #2 but are keeping #1, and I'm curious why. That's genuine curiosity, and a real question, not a thinly-veiled scowl of disapproval disguised as a question :) Anyway, I'm happy enough so long as Raymond's suggested Context.create_decimal() exists, that's the actual functionality I'm after, so maybe I should let you guys get on with it. Thanks. |
|||

msg78846 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-02 17:28 | |

> It looks like you and Raymond have rejected #2 but are keeping #1 I'm not against #2, but I'm not particularly for it either. In any case, once you've converted your float to Decimal it's trivial to round it to whatever precision you feel like, so #2 seems unnecessary to me. -0.0. I am -1.100000000000000088817841970012523233890533447265625 on any implementation of from_float (with or without keywords, defaults, etc.) for which Decimal.from_float(1.1) gives Decimal('1.1'). |
|||

msg78875 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-02 20:44 | |

> once you've converted your float to Decimal it's trivial > to round it to whatever precision you feel like, so #2 > seems unnecessary to me. Agree with Mark on how to control rounding. The DecimalWay(tm) is that used by Context.create_decimal(). A second argument is problematic for two reasons. First, it demands that some rounding take place but doesn't let you control the rounding method or signal an Inexact conversion -- you need a context for that. Second, the API for decimal is already somewhat complex and we don't want to make it worse by introducing a new variant with a second argument that has no parallel elsewhere in the API. Also agree with Mark regarding Decimal.from_float() which needs to be lossless and exact. It parallels fractions.from_float() which does the same thing for rationals. The DecimalWay(tm) is to have Decimal constructors be exact and to use Context based constructors when rounding is desired. I'll submit a patch to this effect (unless someone beats me to it). |
|||

msg78931 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-03 04:33 | |

See attached patch for Py27. |
|||

msg78942 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-03 10:40 | |

Instead of the repeated divisions and Inexact tests, how about a direct approach: n/2**k = (n*5**k)/10**k, so something like: sign = 0 if copysign(1.0, self) == 1.0 else 1 n, d = abs(self).as_integer_ratio() k = d.bit_length() - 1 return _dec_from_triple(sign, str(n*5**k), -k) should work, and would likely be faster too. I also think the sign of 0 should be preserved: i.e., >>> Decimal.from_float(-0.0) Decimal('-0') Am still reviewing---more comments to come. |
|||

msg78946 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-03 11:19 | |

A couple more things: 1. There's a typo 'equilvalent' in the decimal.py part of the patch. 2. Can I suggest using return d._fix(self) instead of return self.plus(d) in create_decimal_from_float. The plus method does two things: rounds to the current context *and* converts -0.0 to 0.0; we only want the first of these. (It's always been a source of mild irritation to me that there's no public method for simply rounding a Decimal to a given context---i.e., a public equivalent of _fix.) |
|||

msg78993 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-03 19:21 | |

The direct method is *much* faster! Applied Mark's suggestions. Committed as r68208 and r68211 . |
|||

msg79045 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-04 11:31 | |

Raymond, Do you think it would be worth replacing the two uses of conditional expressions in Decimal.from_float with if-else statements? Alex Goretoy pointed out (on c.l.p) that the trunk version of decimal.py can no longer be imported into Python 2.4 (and 2.3). I don't know how much this matters, but it seems to go against the comments about 2.3 compatibility at the top of decimal.py. I admit that I don't really understand the motivation for these comments, or whether they're still relevant 4 versions on from Python 2.3. Of course, from_float still won't work with earlier versions of Python, but having one Decimal method unavailable seems like a lesser crime than making 'import decimal' fail. |
|||

msg79084 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-04 20:20 | |

> Do you think it would be worth replacing the two uses of > conditional expressions in Decimal.from_float with if-else > statements? Yes, please. > Of course, from_float still won't work with earlier versions > of Python, but having one Decimal method unavailable seems > like a lesser crime than making 'import decimal' fail. Right. I don't see an easy way around that short of having a conditional compilation, allowing use of alternative slow code multiplying the float repeatedly by two to build-up the float digits. |
|||

msg79579 - (view) | Author: Facundo Batista (facundobatista) * | Date: 2009-01-10 21:43 | |

Raymond, Mark, thanks for this work! I'd include the following in the PEP (under the "from float" discussion), what do you think? """ Update: The .from_float() method was added to Python 2.7 and 3.1 versions, providing lossless and exact conversion from float to Decimal (see issue 4796 [7]_ for further information). It has the following syntax:: Decimal.from_float(floatNumber) where ``floatNumber`` is the float number origin of the construction. Example:: >>> Decimal.from_float(1.1) Decimal('1.100000000000000088817841970012523233890533447265625') """ |
|||

msg79580 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2009-01-10 22:00 | |

I agree the PEP should be updated. Your proposed update looks good to me. |
|||

msg79588 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2009-01-11 05:03 | |

Go ahead with the update, but it isn't really necessary. The PEPs are for getting something into the language in the first place and for centralizing a major discussion. They typically get out of date quickly after they've been implemented. In this case, we've got the tracker discussion to document the minor discussions than led to this minor change in the API. So, it's okay to update the PEP or not. |

History | |||
---|---|---|---|

Date | User | Action | Args |

2009-01-11 05:03:48 | rhettinger | set | messages: + msg79588 |

2009-01-10 22:00:36 | mark.dickinson | set | messages: + msg79580 |

2009-01-10 21:43:11 | facundobatista | set | nosy:
+ facundobatista messages: + msg79579 |

2009-01-04 20:20:15 | rhettinger | set | messages: + msg79084 |

2009-01-04 11:31:56 | mark.dickinson | set | messages: + msg79045 |

2009-01-03 19:21:03 | rhettinger | set | status: open -> closed resolution: fixed messages: + msg78993 versions: + Python 3.1, Python 2.7 |

2009-01-03 11:19:29 | mark.dickinson | set | assignee: mark.dickinson -> rhettinger messages: + msg78946 |

2009-01-03 10:40:42 | mark.dickinson | set | messages: + msg78942 |

2009-01-03 09:38:47 | rhettinger | set | files: - decimal.diff |

2009-01-03 09:38:40 | rhettinger | set | keywords:
+ needs review, - patch assignee: rhettinger -> mark.dickinson files: + decimal2.diff |

2009-01-03 04:33:24 | rhettinger | set | files:
+ decimal.diff keywords: + patch messages: + msg78931 |

2009-01-02 20:44:14 | rhettinger | set | messages: + msg78875 |

2009-01-02 17:28:26 | mark.dickinson | set | messages: + msg78846 |

2009-01-02 17:08:25 | steven.daprano | set | messages: + msg78843 |

2009-01-02 14:09:35 | mark.dickinson | set | messages: + msg78796 |

2009-01-02 13:18:08 | steven.daprano | set | messages: + msg78793 |

2009-01-02 13:05:35 | steven.daprano | set | messages: + msg78789 |

2009-01-02 11:48:37 | mark.dickinson | set | messages: + msg78784 |

2009-01-02 11:31:38 | mark.dickinson | set | nosy:
+ mark.dickinson messages: + msg78781 |

2009-01-01 00:43:08 | rhettinger | set | messages: + msg78671 |

2009-01-01 00:38:23 | rhettinger | set | messages: + msg78670 |

2008-12-31 23:06:05 | rhettinger | set | assignee: rhettinger nosy: + rhettinger |

2008-12-31 22:59:19 | steven.daprano | create |