classification
Title: Floor divide should return int
Type: behavior Stage: needs patch
Components: Interpreter Core Versions: Python 3.5
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, alex, belopolsky, casevh, mark.dickinson, petr.viktorin, pitrou, rhettinger, skrah, tim.peters
Priority: normal Keywords:

Created on 2014-09-19 16:56 by belopolsky, last changed 2015-10-02 21:25 by belopolsky. This issue is now closed.

Messages (24)
msg227107 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-19 16:56
PEP 3141 defines floor division as floor(x/y) and specifies that floor() should return int type.  Builtin float type has been made part of the PEP 3141 numerical tower, but floor division of two floats still results in a float.


See also:

  * #1656 - Make math.{floor,ceil}(float) return ints per PEP 3141
  * #1623 - Implement PEP-3141 for Decimal
  * https://mail.python.org/pipermail/python-ideas/2014-September/029392.html
msg227149 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-20 09:30
Is this change compelling enough to break compatibility, or is it just a matter of purity?
msg227151 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-09-20 10:49
Perhaps it's worth mentioning that several people on Python-ideas took the
opposite view:  math.floor() should return a float.

PEP-3141 does not mention Infinities and NaNs:

"The Real ABC indicates that the value is on the real line, and supports
 the operations of the float builtin. Real numbers are totally ordered
 except for NaNs (which this PEP basically ignores)."

Floats, however, are on the extended real number line, so we have a problem. :)

Other languages
===============

The PEP says that inspiration came from Scheme and Haskell.

However, Scheme returns floats:
-------------------------------

https://mail.python.org/pipermail/python-ideas/2014-September/029432.html

Haskell seems to return the highest representable integer:
----------------------------------------------------------

Prelude> floor (1/0)
179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216

However, Haskell float support looks sketchy:
---------------------------------------------

Prelude> floor (0/0)
-269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824

Prelude> let x = 1 / 0
Prelude> x
Infinity
Prelude> x / 0
Infinity

Considering the last two examples, I think Haskell should not provide any
guidance here. ;)
msg227152 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-09-20 10:54
Argh, forget the second Haskell example: inf / 0 is fine.
msg227156 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-20 16:01
> Is this change compelling enough to break compatibility,
> or is it just a matter of purity?

According to the PEP 3141, Integer is a subtype of Real, so one should be able to substitute an Integer whenever Real is expected.  The reverse is not true, for example

>>> [1,2][1.0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not float

This is one of the common uses of floor division - to find an index of a cell in a regular grid: (x - start) // step.  In this situation, it is convenient to have the result ready to be used as an index without a cast.

The substitution principle also suggests that compatibility issues are likely to be small: in most contexts integers behave the same as floats or "better".

Here is an example of a "better" behavior:

>>> x = 1 + 10**50
>>> x * 1 == x
True
>>> x * 1.0 == x
False

The only case I can think of where float result may be preferable is inf // 1 because integers cannot represent infinity.  However, this case is arguably already broken.

What are the use-cases for float // float where integer result is not acceptable?
msg227158 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-20 16:06
> However, Scheme returns floats

Does Scheme's default integer type support arbitrarily large values?
msg227159 - (view) Author: Case Van Horsen (casevh) Date: 2014-09-20 16:23
>
> Does Scheme's default integer type support arbitrarily large values?
>
Yes, at least is does on the version I tested.
msg227161 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-20 16:38
> Perhaps it's worth mentioning that several people on Python-ideas
> took the opposite view:  math.floor() should return a float.

I sympathize with the idea that math module functions should return floats.  I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()).  It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library.

Note that math module provides the means to compute C-style floor:

>>> x = float('inf')
>>> math.modf(x)[1]
inf
>>> x = -3.4
>>> math.modf(x)[1]
-3.0

Maybe we should add floorf, ceilf, etc. as well.  This, however, is a different issue from the one at hand here.
msg227162 - (view) Author: Case Van Horsen (casevh) Date: 2014-09-20 16:39
> What are the use-cases for float // float where integer result is not acceptable?

It can lead to unexpected memory consumption when dealing with
arbitrary precision values. What should Decimal('1e123456')//1 return?
The result is exactly equal to Decimal('1e123456') but the
corresponding Python integer will consume ~55KB of RAM.

I'm also concerned that returning a very large integer will lead users
to assume that the result is more precise than it really is. Assuming
standard 64-bit double format, only the first 53 bits are significant.
All the remaining bits are 0.

>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue22444>
> _______________________________________
msg227163 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-20 16:44
> What should Decimal('1e123456')//1 return?

I think Decimal case should be considered separately.  Note that unlike float, they are not part of the numerical tower, so PEP 3141 arguments don't apply:

>>> isinstance(1.0, numbers.Real)
True
>>> isinstance(decimal.Decimal(1), numbers.Real)
False
msg227164 - (view) Author: Case Van Horsen (casevh) Date: 2014-09-20 17:05
On Sat, Sep 20, 2014 at 9:38 AM, Alexander Belopolsky
<report@bugs.python.org> wrote:
>
> Alexander Belopolsky added the comment:
>
>> Perhaps it's worth mentioning that several people on Python-ideas
>> took the opposite view:  math.floor() should return a float.
>
> I sympathize with the idea that math module functions should return floats.  I find it unfortunate that math.floor delegates to the __floor__ dunder on non-floats instead of doing math.floor(x.__float__()).  It would be more natural to have a floor builtin that *always* delegates to __floor__ and keep math a pure float library.

+1

>
> Note that math module provides the means to compute C-style floor:
>
>>>> x = float('inf')
>>>> math.modf(x)[1]
> inf
>>>> x = -3.4
>>>> math.modf(x)[1]
> -3.0

That's not immediately obvious...

>
> Maybe we should add floorf, ceilf, etc. as well.  This, however, is a different issue from the one at hand here.
>

i think the issues are related. PEP-3141 defines x//y as
int(floor(x/y)). It also defines divmod(x, y) as (x//y, x % y). These
definitions cannot all be satisfied at the same  Python's divmod
function takes extra effort to calculate x//y precisely. Those
corrections are not possible via floor().

I maintain gmpy2 which wraps the GMP, MPFR, and MPC arbitrary
precision libraries. I originally implemented x//y as floor(x/y). That
choice lead to errors in divmod() that I've fixed in the development
version. I still need to fix floor division: do I make it compatible
with divmod() or floor()?

My preference would be to define floor division and divmod in terms of
each other and allow math.ceil()/floor()/trunc() to return floating
point values. The current definitions are impossible to satisfy.

I mentioned my concerns about memory growth in another comment. I'm
not as concerned about the unexpected memory growth in floor division
as I am in floor() etc.
msg227165 - (view) Author: Case Van Horsen (casevh) Date: 2014-09-20 17:13
>> What should Decimal('1e123456')//1 return?
>
> I think Decimal case should be considered separately.  Note that unlike float, they are not part of the numerical tower, so PEP 3141 arguments don't apply:
>
>>>> isinstance(1.0, numbers.Real)
> True
>>>> isinstance(decimal.Decimal(1), numbers.Real)
> False
>
I maintain gmpy2 and I've had requests to support the numeric tower.
gmpy2 has integral, rational, real, and complex types so I should be
able to.
msg227166 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-20 17:49
> What are the use-cases for float // float where integer result is not acceptable?

I can't think of any. I was mostly making the case for conservatism here.

The indexing use case is interesting, although I suppose enumerate() should eliminate most instances of it.
msg227172 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-09-20 20:14
> Is this change compelling enough to break compatibility,
> or is it just a matter of purity?

I agree with Antoine that making this change is a really bad idea.

1) The current behavior has been around for a long time and is implemented in several modules including decimal and fractions.   As core devs, we need to keep focused on a priority of making the language stable (not making changes that truly necessary and invalidating all previously published material) and more importantly not adding yet more obstacles to converting from Python 2 to Python 3 (which Guido has called "death by a thousand cuts").

2) The current behavior can be useful it that it allows floor division operations without unexpected type conversions occurring in the middle of an expression.  We really don't want to break those use cases.

# Here is a simple example of a chain of calculations 
# where preserving the type matters

from __future__ import print_function
from fractions import Fraction
from decimal import Decimal

def f(x, y):
    return x // 3 * 5 / 7 + y

def g(x, y):
    return int(x // 3) * 5 / 7 + y

for x, y in [
        (Fraction(85, 7), Fraction(2, 3)),
        (Decimal('12.143'), Decimal('0.667')),
        (12.143, 0.667),
    ]:
    print(f(x, y), g(x, y))

In Python 2:
------------
8/3 8/3
3.524142857142857142857142857 2.667
3.52414285714 2.667

In Python 3:
------------
3.5238095238095237 3.5238095238095237
Traceback (most recent call last):
  ...
    return int(x // 3) * 5 / 7 + y
TypeError: unsupported operand type(s) for +: 'float' and 'decimal.Decimal'


I am a strong -1 against breaking code that relies on the floor division being type preserving.

The PEP should be revised to say that floor division is defined to return a value that is *equal* to an Integral but not place any restriction on the return type.
msg227180 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2014-09-20 21:48
Floor division on floats is an unattractive nuisance and should be removed, period - so there ;-)

But short of that, I favor leaving it alone.  Whose life would be improved by changing it to return an int?  Not mine - and doing so anyway is bound to break existing code.  +1 on changing the PEP instead (as Raymond suggested).
msg227183 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2014-09-20 22:16
I can't say that I've ever used // on floats, but it seems to me anyone doing so (as opposed to normal division + explicit rounding) *intentionally* might be broken by this change, but anyone doing this incidentally is not really in a "gotcha" situation. Since this is a type-specific behavior, and not a value-specific one, I don't really think there's a win in changing the behavior, and staying backwards compatible is much better.
msg227277 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-22 15:39
[Raymond]
> The current behavior has been around for a long time and is implemented in several modules including decimal and fractions.

No, in the fractions module floor division returns an int:

>>> type(Fraction(2) // Fraction(1))
<class 'int'>

It is also implemented in the datetime module where

>>> type(timedelta(2) // timedelta(1))
<class 'int'>


[Raymond]
# Here is a simple example of a chain of calculations 
# where preserving the type matters
..

def f(x, y):
    return x // 3 * 5 / 7 + y

def g(x, y):
    return int(x // 3) * 5 / 7 + y
[/Raymond]

I am not sure what is the problem here.  In Python 3:

>>> f(12.143, 0.667)
3.5241428571428575
>>> g(12.143, 0.667)
3.5241428571428575
msg227279 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-09-22 17:00
-1 from me, too.  It's an unnecessary change, and the conversion from float to integer potentially expensive compared to the computation of the floating-point result (especially in extended floating-point implementations that allow a wider exponent range).

If this is about consistency between `//` and `math.floor`, I'd much rather see `math.floor` go back to returning a float instead of an int.
msg227281 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-22 17:23
[Raymond]
> The PEP should be revised to say that floor division is defined to 
> return a value that is *equal* to an Integral but not place any
> restriction on the return type.

If we take this route, what float('inf') // 1 and float('nan') // 1 should return?
msg227282 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-09-22 17:29
> If we take this route, what float('inf') // 1 and float('nan') // 1 should return?

Probably exactly the same as they do right now.  I think there's an argument that `float('inf') // 1` "should have been" `float('inf')`.  But I'm not sure there's much of a case for *changing* the current return value.
msg227301 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-22 19:40
Mark,

Raymond suggested that "The PEP 3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral".

Since nan or inf are not *equal* to any Integral, the current implementation does not comply.  In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent:

>>> Decimal('inf') // 1
Decimal('Infinity')
>>> float('inf') // 1
nan
msg227303 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-09-22 19:48
Alexander Belopolsky <report@bugs.python.org> wrote:
> Raymond suggested that "The PEP 3141 should be revised to say that floor division is defined to return a value that is *equal* to an Integral".

I guess it should say "equal to an Integral or a special value".

> Since nan or inf are not *equal* to any Integral, the current implementation does not comply.  In the absence of a recommendation in the PEP, implementers of new numeric types are left with little guidance because existing types are inconsistent:
> 
> >>> Decimal('inf') // 1
> Decimal('Infinity')
> >>> float('inf') // 1
> nan

I think both should return inf.
msg227305 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2014-09-22 19:55
skrah> I think both should return inf.

What about this case:

>>> Decimal('1') // Decimal('-inf')
Decimal('-0')
>>> 1. // float('-inf')
-1.0
msg227306 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2014-09-22 20:02
I think that one's covered by #22198.
History
Date User Action Args
2015-10-02 21:25:03belopolskysetstatus: open -> closed
resolution: rejected
2014-09-26 18:37:00petr.viktorinsetnosy: + petr.viktorin
2014-09-22 20:02:33mark.dickinsonsetmessages: + msg227306
2014-09-22 19:55:13belopolskysetmessages: + msg227305
2014-09-22 19:48:40skrahsetmessages: + msg227303
2014-09-22 19:40:43belopolskysetmessages: + msg227301
2014-09-22 17:29:15mark.dickinsonsetmessages: + msg227282
2014-09-22 17:23:40belopolskysetmessages: + msg227281
2014-09-22 17:00:16mark.dickinsonsetnosy: + mark.dickinson
messages: + msg227279
2014-09-22 15:39:42belopolskysetmessages: + msg227277
2014-09-21 09:43:42Arfreversetnosy: + Arfrever
2014-09-20 22:16:16alexsetmessages: + msg227183
2014-09-20 21:48:45tim.peterssetmessages: + msg227180
2014-09-20 21:34:25rhettingersetnosy: + alex
2014-09-20 20:15:04rhettingersetnosy: + tim.peters
2014-09-20 20:14:38rhettingersetmessages: + msg227172
2014-09-20 17:49:47pitrousetmessages: + msg227166
2014-09-20 17:13:22casevhsetmessages: + msg227165
2014-09-20 17:05:05casevhsetmessages: + msg227164
2014-09-20 16:44:52belopolskysetmessages: + msg227163
2014-09-20 16:39:28casevhsetmessages: + msg227162
2014-09-20 16:38:08belopolskysetmessages: + msg227161
2014-09-20 16:23:13casevhsetmessages: + msg227159
2014-09-20 16:06:29belopolskysetmessages: + msg227158
2014-09-20 16:01:29belopolskysetmessages: + msg227156
2014-09-20 10:54:26skrahsetmessages: + msg227152
2014-09-20 10:49:51skrahsetmessages: + msg227151
2014-09-20 09:30:23pitrousetnosy: + pitrou
messages: + msg227149
2014-09-20 08:17:12rhettingersetmessages: - msg227146
2014-09-20 06:27:32rhettingersetnosy: + rhettinger
messages: + msg227146
2014-09-20 04:25:41casevhsetnosy: + casevh
2014-09-19 18:31:33skrahsetnosy: + skrah
2014-09-19 16:56:55belopolskycreate