classification
Title: Strange results for floor division ("//") with non-integer divisors
Type: behavior Stage:
Components: None Versions: Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: loewis, mark.dickinson, rhettinger, serhiy.storchaka, skrah, tom.pohl
Priority: normal Keywords:

Created on 2012-11-12 08:25 by tom.pohl, last changed 2012-11-14 10:27 by tom.pohl. This issue is now closed.

Messages (19)
msg175424 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-12 08:25
According to the documentation of the floor division (http://docs.python.org/2/reference/expressions.html#binary-arithmetic-operations), x//y should be equal to math.floor(x/y).

However, the result of 1//0.1 is 9.0 (tested on 2.6, 2.7, 3.2).

It might be related to the internal representation of floating-point numbers, but for this example I would expect it to come up with the correct values.

Cheers,
Tom
msg175425 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-12 08:53
Yes, this is related to the internal representation of floating-point numbers.  0.1 is 3602879701896397/36028797018963968 in float.

>>> import fractions
>>> fractions.Fraction(0.1)
Fraction(3602879701896397, 36028797018963968)
>>> 36028797018963968 / 3602879701896397
10.0
>>> 36028797018963968 // 3602879701896397
9
msg175426 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-11-12 08:55
9.0 *is* the correct result here.  The number that Python stores for 0.1 is an approximation that's actually a little greater than 0.1.
msg175428 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-12 09:12
Thanks for your comments. From a technical/numerical point of view I agree with you that the computed result is correct given the floating-point limitations.

From a user's point of view (and the documentation seems to agree with me) the result is wrong. The documentation (see link in first post) says: [...] the result is that of mathematical division with the ‘floor’ function applied to the result.

math.floor(1/0.1) returns the expected 10.0 and the doc says that the floor division should behave the same.

If 9.0 for 1//0.1 is the correct result according to your definition of the floor division then I cannot think of a reasonable use case for the floor division.
msg175430 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-11-12 09:52
Tom: there's no reasonable way to define all 3 of /, // and % for floating-point numbers that avoids all user surprises.  There are a couple of notes (nos 2 and 3) at the bottom of the documentation page you refer to that attempt to explain some of the potential difficulties here.

> I cannot think of a reasonable use case for the floor division.

Indeed, one has to be careful when using floating-point for *anything* where tiny numerical errors can significant:  rounding is another example of this.

What's your application?  There may be better ways of doing what you're trying to do.  If you're working with financial data, you might want to look at the decimal module.  If you're working with Python floats (or floating-point arithmetic in *any* programming language), your code has to be robust in the face of small errors.

> math.floor(1/0.1) returns the expected 10.0

Yep.  What happens here is that the exact result of 1 / 0.1 is just a touch under 10.0, but the closest representable float to that exact result is 10.0 itself.  So the result of the division is rounded up to 10.0, and then the floor returns 10.0

> and the doc says that the floor division should behave the same.

Where do the docs say that?
msg175433 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-11-12 10:05
Tom: you are misinterpreting the docs. It says (paraphrased) that the result of x//y equals floor(x mathematically-divided-by y), which is different from floor(x/y). Your computer is not capable of performing the mathematically-divided-by operation; you have to compute it on paper.
msg175434 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-12 10:21
> Your computer is not capable of performing the mathematically-divided-by operation; you have to compute it on paper.

You can compute it with Python.

>>> math.floor(1/fractions.Fraction(0.1))
9
msg175443 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-12 13:53
Thanks for all the explanations why Python's floor division (FD) works as specified. And I agree, it does work as specified, but still, I think this is not the behavior that most people would expect and is therefore dangerous to provide/use.

What do I expect from FD (x//y):
1. Perform a division (I don't care about the details here).
2. Return an integer value q (even if it's stored in a float).
3. The absolute difference between the mathematical division q'=x/y and the returned result q is less than 1, since it's just a floor operation, right?

1//0.1 = 9.0 violates my personal last expectation and I guess I'm not the only one.

My use case: I need to perform a division, but the method only accepts integer values, so what I used to do is FD. Since today I'm using int(x/y) or floor(x/y).
msg175444 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-12 14:04
Martin:
Ok, just as you suggested, I did the calculations on a sheet of paper:

floor(1 mathematically-divided-by 0.1) = floor(10) = 10

qed ;-)
msg175445 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2012-11-12 14:25
Any programming language that uses binary floats behaves like that
and it is actually what people expect.

If you want behavior that is closer to pencil and paper calculations,
you need to use decimal:

>>> Decimal(1) // Decimal("0.1")
Decimal('10')


Contrast with:

>>> Decimal(1) // Decimal(0.1)
Decimal('9')


The reason:
>>> Decimal("0.1")
Decimal('0.1')


>>> Decimal(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
msg175463 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-12 16:42
Since nobody seems to share my point of view, I give up. :-)

Thanks for your support and for working on Python (the best programming language as we all know),
Tom
msg175495 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-11-13 10:18
Am 12.11.12 14:53, schrieb Tom Pohl:
> What do I expect from FD (x//y):
> 1. Perform a division (I don't care about the details here).
> 2. Return an integer value q (even if it's stored in a float).
> 3. The absolute difference between the mathematical division q'=x/y and the returned result q is less than 1, since it's just a floor operation, right?
>
> 1//0.1 = 9.0 violates my personal last expectation and I guess I'm not the only one.

However, it matches *precisely* your description of what you expect:

1. 1 divided-by 0.1 gives
9.99999999999999944488848768742176060306327615036178207623262235371852234374499248...

Please understand that the correct result of 1 divided-by 0.1 is *not* 
10, because 0.1 is *not* 1/10.

2. returned is then the integer 9

3. the difference is
.99999999999999944488848768742176060306327615036178207623262235371852234374499248...
which is indeed smaller than 1.

P.S. In case you want to get a more exact result of 1 divided-by 0.1, 
get the digits from

int(1/fractions.Fraction(0.1)*10**180)
msg175537 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-14 06:35
You still get me wrong. Thanks to your explanations and to my personal knowledge about this topic (programming for 28 years now; PhD in CS/numerics/HPC) I now fully understand the technical details about the current implementation of FD. The problem I see is that the average Python user does and should not care about such details.

I talked to a bunch of people (n=7) here at the company where I also give Python courses from time to time. I asked them two questions:
1. Is this behavior of FD what you would expect?
2. Given the current behavior of FD, what use cases do you see?

The answers were always the same (and I tend to agree):
1. No.
2. No idea.

All of you seem to answer the first questions with yes, but what's your answer to the second question? What would you recommend your 10-year-old son or your 62-year-old mother to do with the current FD operator?
msg175538 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2012-11-14 06:53
FWIW, I agree with this being closed.  The current behavior is a fact-of-life for anyone using binary floating point.   

The decimal module is provided people who want more intuitive behaviors when dividing by numbers like 0.1 which are exactly representable as a decimal fraction but not as a binary fraction.
msg175542 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-14 07:11
This is "a fact-of-life for anyone using binary floating point":
x = 0.0
while x != 1.0: print(x); x += 0.1  # so long


This is not: 1 // 0.1 = 9.0 because math.floor(1/0.1) is able to come up with the result that is expected from an operator called "floor division".

Aynway, I'm curios, what's your use case for FD? Since all of you are so strongly in favor of the current behavior you must be using it all the time.
msg175556 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-11-14 09:54
> I talked to a bunch of people (n=7) here at the company where I also  
> give Python courses from time to time. I asked them two questions:
> 1. Is this behavior of FD what you would expect?
> 2. Given the current behavior of FD, what use cases do you see?
>
> The answers were always the same (and I tend to agree):
> 1. No.
> 2. No idea.
>
> All of you seem to answer the first questions with yes, but what's  
> your answer to the second question?

It's not that I had expected that answer, and I certainly agree that it
is confusing. However, I also believe that it is the correct answer.

> What would you recommend your 10-year-old son or your 62-year-old  
> mother to do with the current FD operator?

The most obvious use case for FD is integer division. Neither my son nor
my mom should use it for floating point (nor should they use floating-point
in the first place).

It's floating point. *Of course* it misbehaves.
msg175557 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-11-14 10:01
Zitat von Tom Pohl <report@bugs.python.org>:

> This is not: 1 // 0.1 = 9.0 because math.floor(1/0.1) is able to  
> come up with the result that is expected from an operator called  
> "floor division".

You apparently assume that it is possible to give a definition to FD
for floating point that is less confusing. I do not think that this
is possible; in particular, I believe that definining x//y as
math.floor(x/y) is also confusing, in other cases (without being able
to construct such cases right away).
msg175558 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-11-14 10:11
> I believe that definining x//y as math.floor(x/y) is also confusing
> in other cases (without being able to construct such cases right away).

In addition, defining x//y as math.floor(x / y) would break its connection with %:  a key invariant is that

    (x // y) * y + x % y should be (approximately in the case of floats) equal to y.

The connection between // and % is more fundamental than the connection between // and /, so in cases where the two disagree, the %-related one wins.

For applications:  it's true that they're not common, but they do exist.  One such is argument reduction:  e.g., for a toy case, suppose that you're implementing a function that computes and returns sin and cos.  The computation can be reduced to computing for angles between 0 and pi / 4:

def sincos(x):
    """ Compute and return sin(x) and cos(x). """
    q, r = divmod(x, pi / 4)
    <compute sincos(r)>
    <use symmetries and the last 3 bits of q to compute sincos(x)>

This is an example where if the relationship between % and // were broken, we'd get wrong results---not simply inaccurate, but completely wrong.

It's also worth noting that // and % are special in that they're the only basic arithmetic operations that can be computed *exactly*, with no numeric error, for a wide range of inputs:  e.g., if x and y are positive and x / y < 2**53, then both x // y and x % y return exact results.  Modifying them to return inexact results instead would be ... surprising.
msg175559 - (view) Author: Tom Pohl (tom.pohl) Date: 2012-11-14 10:27
Mark, thanks for explaining the connection of // and %. Finally, I can see why somebody would want to stick to the current behavior of FD.

It renders FD useless for all of my use cases, but there are simple alternatives.

Thanks for your time,
Tom
History
Date User Action Args
2012-11-14 10:27:26tom.pohlsetmessages: + msg175559
2012-11-14 10:11:41mark.dickinsonsetmessages: + msg175558
2012-11-14 10:01:06loewissetmessages: + msg175557
2012-11-14 09:54:26loewissetmessages: + msg175556
2012-11-14 07:11:45tom.pohlsetmessages: + msg175542
2012-11-14 06:53:47rhettingersetnosy: + rhettinger
messages: + msg175538
2012-11-14 06:35:44tom.pohlsetmessages: + msg175537
2012-11-13 10:18:49loewissetmessages: + msg175495
2012-11-12 16:42:24tom.pohlsetmessages: + msg175463
2012-11-12 14:25:12skrahsetmessages: + msg175445
2012-11-12 14:04:41tom.pohlsetmessages: + msg175444
2012-11-12 13:53:41tom.pohlsetmessages: + msg175443
2012-11-12 10:21:40serhiy.storchakasetmessages: + msg175434
2012-11-12 10:05:43loewissetnosy: + loewis
messages: + msg175433
2012-11-12 09:52:21mark.dickinsonsetmessages: + msg175430
2012-11-12 09:12:58tom.pohlsetmessages: + msg175428
2012-11-12 08:55:54mark.dickinsonsetstatus: open -> closed
2012-11-12 08:55:48mark.dickinsonsetresolution: not a bug
messages: + msg175426
2012-11-12 08:53:09serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg175425
2012-11-12 08:33:39ezio.melottisetnosy: + mark.dickinson, skrah
2012-11-12 08:25:05tom.pohlcreate