Issue4087

Created on **2008-10-09 13:39** by **mark.dickinson**, last changed **2008-11-21 09:30** by **mark.dickinson**. This issue is now **closed**.

Messages (10) | |||
---|---|---|---|

msg74576 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2008-10-09 13:39 | |

The Decimal module breaks transitivity of equality: Decimal(2) == 2 and 2 == float(2), but Decimal(2) != float(2). The Python set and dict implementations rely on transitivity of equality for correct operation. These two facts together give some strange results when playing with sets and dicts involving Decimals and floats. For example (with Python 2.6): >>> s = set([Decimal(2), float(2)]) >>> t = set([2]) >>> s | t == t | s False >>> len(s | t) 2 >>> len(t | s) 1 Other strange examples, and possible solutions, were discussed recently on comp.lang.python; see the thread starting at: http://mail.python.org/pipermail/python-list/2008-September/508859.html Possible solutions: (1) Document the problem, making it clear in the language reference that correct set operation relies on transitivity of equality, and adding a note to the decimal documentation indicating that mixing floats and Decimals in a container is asking for trouble. (2) Fix up Decimal so that equal numeric objects compare equal; this would also involve fixing the hash operation. To me, this goes against the philosophy of keeping the Decimal module close to the specification. (3) Terry Reedy suggested (in the c.l.python thread linked to above) a simpler version of (2): allow Decimal(i) == float(i) or Decimal(i) == Fraction(i) to return True for all integers i. (Decimal('0.5') == 0.5 would still return False.) This fixes transitivity, and has the advantage of not requiring any changes to the hash functions: hash(Decimal(i)) == hash(float(i)) is already true for all integers i. My own preference would be simply to document the problem; it doesn't seem like something that's going to affect that vast majority of Python users. Raymond, Facundo: any thoughts? |
|||

msg74586 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2008-10-09 16:45 | |

There are two issues involved: 1. documenting set behavior 2. what to do, if anything, about Decimals and other numbers Since users are free to create similar problems, and since sets are missing from the Reference section on comparisons, I opened a separate set documentation issue http://bugs.python.org/issue4090 leaving this as a Decimal-other_number equality issue. The root of the problem is that all members of s are members of t and vice versa. This should make s and t equal, but they are not. This also breaks the definitions of issubset (<=), issuperset (>=), union (|), and symmetric_difference (^) as shown in the c.l.p thread. Transitivity is also fundamental in logic and the rule of substitution. So I strongly prefer that it be preserved in Python as released. Another way to restore transitivity is (4) make Decimal(1) != int(1) just as Decimal(1) != float(1). Having a Decimal be equal in value to just one of two things that are equal in value is incoherent, and leads to incoherent results. |
|||

msg74587 - (view) | Author: Facundo Batista (facundobatista) * | Date: 2008-10-09 17:05 | |

(Ok, remember that I'm not a "numeric" guy before start hitting me, :p ) I think that if we have Decimal(1)==1, and 1==1.0, to have Decimal(1)==1.0. We always rejected comparison with "unsupported types", but having this situation, I'd propose to put something like the following at the beggining of __eq__() and similars: def __eq__(self, other): if isinstance(other, float) and int(other)==other: other = int(other) What do you think? |
|||

msg74588 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2008-10-09 17:13 | |

Recommend not "doing anything" about decimals and other numbers. What you're seeing is a predictable consequence of NotImplemented being returned by some but not all cross type comparisons. IMO, it is perfectly reasonable that both decimals and floats can be compared to integers but not to each other. Integers are a "universal donor" in this respect but the two float types are not. It is true that equality should be transitive but the same cannot be said for the *ability of types* to be compared. Unfortunately, the == operator masks what is going on by returning False instead of raising a NotImplementedError. IOW, it the apparant loss of transitivity when "float(2) == Decimal(2)" returns False is an illusion; instead, the False return means that the types cannot be compared at all. If any doc changes are made with respect to this issue, it should be in the docs for the == and != operators and for NotImplemented. |
|||

msg74590 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2008-10-09 18:06 | |

If Decimal(2) == float(2) were to raise an error, set([Decimal(2), float(2)]) would fail, as I would argue it ought to, and the set anomalies would disappear. |
|||

msg74612 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2008-10-10 00:20 | |

I'll take up the doc fix for the current state of affairs. A change from returning NotImplemented to raising NotImplementedError would need be a separate feature request or PEP. Alternatively, we could decide to allow decimal/float comparisons -- the float can be converted to a decimal exactly and compared exactly -- it would be slow but it would work and have precise semantics. Am resetting the priority back to low because the current behavior is exactly what is supposed to happen. It is an automatic consequence of how we use NotImplemented and the decision to allow integer/float and integer/decimal comparisons but not float/decimal comparisons. There is not bug here, just something that offends ones sensibilities. |
|||

msg74631 - (view) | Author: Facundo Batista (facundobatista) * | Date: 2008-10-10 12:21 | |

2008/10/9 Raymond Hettinger <report@bugs.python.org>: > Alternatively, we could decide to allow decimal/float > comparisons -- the float can be converted to a decimal > exactly and compared exactly -- it would be slow but > it would work and have precise semantics. -0 Note that this could lead to surprising behaviours, when doing these comparations... Decimal("1.1")==1.1 will be true, buy maybe Decimal("1.235445687")==1.235445687 will not (I didn't try if this particular comparison will fail, but hope you get the idea). This is why I suggested the other way... we now allow comparison to integers, let's allow comparisons when the floats are equal to the integers, and no more. |
|||

msg74641 - (view) | Author: Terry J. Reedy (terry.reedy) * | Date: 2008-10-10 19:57 | |

More sensibility offenders: >>> s = {fractions.Fraction(17,1), decimal.Decimal(17)} >>> s-{17} set() Removing one thing removes two. >>> s.remove(17) >>> 17 in s True Removing something leaves it there. >>> s {Fraction(17, 1)} # random choice Removing one thing and subtracting the set with that one thing give different results. >>> s = {decimal.Decimal(17), fractions.Fraction(17,1)} >>> s.remove(17) >>> s {Decimal('17')} The behavior of 'set' s depends on the order items are added. > Facundo's suggested code: if isinstance(other, float) and int(other)==other: other = int(other) would be more efficient, I assume as if isinstance(other,float): ifloat = int(other) if other == ifloat: other = ifloat or if the CAPI has an efficient 'float_isint' function that accesses the bits of a float, as the C equivalent of if isinstance(other, float) and float_isint(other): other = int(other) I remember float-Decimal comparison being rejected/deferred in part for being problematical for fractional values. That is why I suggested implementing it, at least for the present, for integral floats (and Fractions) only, which are relatively easy to detect. |
|||

msg75990 - (view) | Author: Raymond Hettinger (rhettinger) * | Date: 2008-11-17 22:56 | |

r67249 |
|||

msg76168 - (view) | Author: Mark Dickinson (mark.dickinson) * | Date: 2008-11-21 09:30 | |

Looks good. Thanks, Raymond. Minor typos: "numberic" -> "numeric" "the expression ``x in y`` equivalent to" -> "the expression ``x in y`` is equivalent to" |

History | |||
---|---|---|---|

Date | User | Action | Args |

2008-11-21 09:30:32 | mark.dickinson | set | messages: + msg76168 |

2008-11-17 22:56:43 | rhettinger | set | status: open -> closed resolution: fixed messages: + msg75990 |

2008-10-10 19:57:33 | terry.reedy | set | messages: + msg74641 |

2008-10-10 12:21:20 | facundobatista | set | messages: + msg74631 |

2008-10-10 00:20:07 | rhettinger | set | priority: normal -> low assignee: rhettinger messages: + msg74612 |

2008-10-09 18:06:28 | terry.reedy | set | priority: low -> normal assignee: georg.brandl -> (no value) messages: + msg74590 |

2008-10-09 17:13:46 | rhettinger | set | title: equality involving Decimals is not transitive; strange set behaviour results -> Document the effects of NotImplemented on == and != nosy: + georg.brandl messages: + msg74588 priority: normal -> low assignee: georg.brandl components: + Documentation |

2008-10-09 17:05:05 | facundobatista | set | messages: + msg74587 |

2008-10-09 16:45:09 | terry.reedy | set | messages: + msg74586 |

2008-10-09 13:39:43 | mark.dickinson | create |