classification
Title: type() constructor should bind __int__ to __index__ when __index__ is defined and __int__ is not
Type: enhancement Stage: test needed
Components: Versions: Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, amitava.b, benjamin.peterson, josh.r, r.david.murray, terry.reedy, trent
Priority: low Keywords:

Created on 2013-12-28 19:56 by ethan.furman, last changed 2018-03-05 21:16 by josh.r.

Files
File name Uploaded Description Edit
test_index_not_int.py amitava.b, 2014-06-22 20:50
Messages (17)
msg207048 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2013-12-28 19:56
In order to create a coherent integer type class both __int__ and __index__ must be defined or the resulting instances will behave inconsistently in different places.

For example, if __index__ is not defined then the class cannot be used in slices, and if __int__ is not defined then int(integer_type) will fail.

At this point the programmer must remember to define both, but since they should return the same value we can have the type constructor automatically assign __int__ to __index__ when the latter is defined and the former is not.  This would be similar to how we currently treat __ne__ when __eq__ is defined.
msg207050 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-12-28 20:08
__ne__ is not "bound" to __eq__. The comparison code simply falls back onto __eq__ if __ne__ is not defined.
msg207051 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2013-12-28 20:23
True.  I meant similar in that Python will use what's available to fill in the blank:

  class has __eq__ but user tried != ?  use __eq__ and invert result

  class has __index__ but user tried int() ?  use __index__ as-is
msg207065 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-12-29 02:19
It is not clear to me that that would be correct, though.  Isn't the whole point of __index__ that some types can act as indicies even though they are *not* integers?  Shouldn't it be up to the type to decide if converting them to int is a sensible thing to do?  Maybe that's being silly/pendantic, though.  On the gripping hand, it feels like this is another case of what you have pointed out elsewhere, that it is not clear that we actually have a consistent API here...
msg207068 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2013-12-29 08:11
In issue19995, in msg206339, Guido exhorted:
--------------------------------------------
>> [Ethan claimed] it is possible to want a type that can be used as an
>> index or slice but that is still not a number
>
> I'm sorry, but this requirement is absurd. An index *is* a number. You
> have to make up your mind. (I know, in the context of the example that
> started this, this is funny, but I still stand by it.)
>
> Finally, the correct name should perhaps have been __integer__ but I don't
> see enough reason to change it now.

The de facto API that is forming is that if an actual int is needed from an actual integer type (not float, not complex, not etc.), then __index__ is used.  If __index__ is not defined by some numeric type then it will not be considered a true int in certain key places in Python, such as as indices, arguments to hex(), etc.

Making the change suggested in the title would help solidify the API.
msg207072 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-12-29 13:44
Ah, I see.  A link to that issue would have been helpful :).

To summarize for anyone like me who didn't follow that issue: __index__ means the object can be losslessly converted to an int (is a true int), while __int__ may be an approximate conversion.  Thus it makes sense for an object to have an __int__ but not __index__, but vice-versa does not make sense.

Is someone updating the docs to reflect this, or should that be spun off as a separate issue as well?
msg207075 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2013-12-29 14:57
I have the following as part of the patch for that issue:
---------------------------------------------------------
diff -r b668c409c10a Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst	Sat Dec 28 20:37:58 2013 +0100
+++ b/Doc/reference/datamodel.rst	Sun Dec 29 06:55:11 2013 -0800
@@ -2073,23 +2073,31 @@ left undefined.
       builtin: float
       builtin: round
 
    Called to implement the built-in functions :func:`complex`,
    :func:`int`, :func:`float` and :func:`round`.  Should return a value
    of the appropriate type.
 
 
 .. method:: object.__index__(self)
 
-   Called to implement :func:`operator.index`.  Also called whenever Python needs
-   an integer object (such as in slicing, or in the built-in :func:`bin`,
-   :func:`hex` and :func:`oct` functions). Must return an integer.
+   Called to implement :func:`operator.index`, and whenever Python needs to
+   losslessly convert the numeric object to an integer object (such as in
+   slicing, or in the built-in :func:`bin`, :func:`hex` and :func:`oct`
+   functions). Presence of this method indicates that the numeric object is
+   an integer type.  Must return an integer.
+
+   .. note::
+
+      When :meth:`__index__` is defined, :meth:`__int__` should also be defined,
+      and both shuld return the same value, in order to have a coherent integer
+      type class.
---------------------------------------------------------

If for some reason that patch doesn't make it into 3.4 I'll split the doc change off to its own issue, unless you think it should be split off anyway?
msg207084 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-12-29 20:06
Nah, splitting it doesn't seem worth it unless you think the patch won't make it in.

(Not that I looked at it earlier, but he patch on the issue doesn't look like what you just posted here...and here there's a typo: shuld).
msg207085 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2013-12-29 20:30
I updated it as I liked your wording better.  :)  Doing more testing to see if anything else needs fixing before I make the next patch for the tracker on that issue.
msg207263 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-01-04 00:33
I believe this is my suggestion 2) in msg206715 of #19995, and I think it better than 3) (and 1) alone). Thanks for moving this forward. I believe what Guido said is that indexes (integers in the broad sense) are (or should be) a subset of things convertible to ints. I concur completely.
msg221310 - (view) Author: Amitava Bhattacharyya (amitava.b) Date: 2014-06-22 20:50
I started working on this issue as part of the Bloomberg Python Sprint (https://etherpad.mozilla.org/LjZPQ55oZs).

Ethan, can you confirm if the attached test case summarizes the issue correctly?

I tried to hook _PyLong_FromNbInt to check nb_index but didn't make progress there. Will need to load in a debugger to see what's going on.
msg221339 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2014-06-23 04:14
Yes, that test should pass when this issue is resolved.

I can't tell from your comment what you are trying to do to resolve it, but the course of action I had in mind was to have the `type` meta-class make a final examination of the type it was creating, and if that type had __index__ but not __int__ then bind __int__ to __index__.
msg221453 - (view) Author: Amitava Bhattacharyya (amitava.b) Date: 2014-06-24 11:56
Hi Ethan, I tried adding a call to `nb_index` (if that slot exists) in `_PyLong_FromNbInt`, but that didn't work. Based on your comment it seems the fix would be to patch this at object creation. I will check where that happens (bear with me while I familiarize myself with the code base :) ).
msg221459 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2014-06-24 13:48
Thank you for your efforts, Amitava.  Please also sign a Contributor's License Agreement so we can actually use your code.  :)

    http://www.python.org/psf/contrib/
msg221474 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-24 16:11
There will be a corporate agreement from Bloomberg sometime soon (and that will be required in this case, not an individual agreement).
msg221510 - (view) Author: Amitava Bhattacharyya (amitava.b) Date: 2014-06-24 23:27
I did fill in the contributor agreement form and e-signed it, maybe it takes some time for my profile to be updated? But I guess I need to wait for the corporate agreement. :)
msg313287 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2018-03-05 21:16
Pingback from #33002, which is caused by the fact that parts of the CPython code base that use PyNumber_Index for type conversion still pre-check for valid types with PyNumber_Check, meaning that a type with __index__ and not __int__ gets erroneously rejected by the pre-check.
History
Date User Action Args
2018-03-05 21:16:06josh.rsetnosy: + josh.r
messages: + msg313287
2015-07-21 07:58:21ethan.furmansetnosy: - ethan.furman
2014-06-24 23:27:35amitava.bsetmessages: + msg221510
2014-06-24 16:11:56r.david.murraysetmessages: + msg221474
2014-06-24 13:48:42ethan.furmansetmessages: + msg221459
2014-06-24 11:56:52amitava.bsetmessages: + msg221453
2014-06-23 04:14:50ethan.furmansetmessages: + msg221339
2014-06-22 20:50:11amitava.bsetfiles: + test_index_not_int.py
nosy: + trent, amitava.b
messages: + msg221310

2014-01-04 00:33:51terry.reedysetstage: test needed
2014-01-04 00:33:35terry.reedysetnosy: + terry.reedy
messages: + msg207263
2013-12-29 20:30:07ethan.furmansetmessages: + msg207085
2013-12-29 20:06:52r.david.murraysetmessages: + msg207084
2013-12-29 14:57:18ethan.furmansetmessages: + msg207075
2013-12-29 13:44:05r.david.murraysetmessages: + msg207072
2013-12-29 08:11:38ethan.furmansetmessages: + msg207068
2013-12-29 02:19:26r.david.murraysetnosy: + r.david.murray
messages: + msg207065
2013-12-28 20:41:45Arfreversetnosy: + Arfrever
2013-12-28 20:23:56ethan.furmansetmessages: + msg207051
2013-12-28 20:08:52benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg207050
2013-12-28 19:56:32ethan.furmancreate