classification
Title: Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gvanrossum, hroncok, mark.dickinson, remi.lapeyre, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2019-02-20 06:43 by serhiy.storchaka, last changed 2019-06-03 10:20 by mark.dickinson. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11952 merged serhiy.storchaka, 2019-02-20 06:54
PR 13740 mark.dickinson, 2019-06-03 10:20
Messages (8)
msg336041 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-20 06:43
Currently, C API functions that convert a Python number to a C integer like PyLong_AsLong() and argument parsing functions like PyArg_ParseTuple() with integer converting format units like 'i' use the __int__() special method for converting objects which are not instances of int or int subclasses. This leads to dropping the fractional part if the object is not integral number (e.g. float, Decimal or Fraction). In some cases, there is a special check for float, but it does not prevent truncation for Decimal, Fraction and other numeric types.

For example:

>>> chr(65.5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: integer argument expected, got float
>>> chr(Decimal('65.5'))
'A'

The proposed PR makes all these functions using __index__() instead of __int__() if available and emit a deprecation warning when __int__() is used for implicit conversion to a C integer.

>>> chr(Decimal('65.5'))
<stdin>:1: DeprecationWarning: an integer is required (got type decimal.Decimal)
'A'

In future versions only __index__() will be used for the implicit conversion, and __int__() will be used only in the int constructor.
msg336050 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-20 08:34
Numerous explicit calls of PyNumber_Index() which are used to protect from passing non-integral types to PyLong_AsLong() and like can be removed after the end of the deprecation period. I tried to mark calls which can be removed with comments, but virtually all calls can be removed.
msg336069 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-02-20 11:10
I like the idea. Rejecting float but not decimal.Decimal is inconsistent. __index__ has been written explicitly for this purpose.

I'm always confused and lost in subtle details of the Python and C API in how they handle numbers, so I wrote some notes for myself:
https://pythondev.readthedocs.io/numbers.html

Some functions accept only int, others use __int__, others __index__. Some functions silently truncate into 32 or 64-bit signed integer. Some other raise a OverflowError or ValueError...
msg336071 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-02-20 11:13
See also bpo-34423: "Overflow when casting from double to time_t, and_PyTime_t" or "How to reduce precision loss when converting arbitrary number to int or float?".
msg336107 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-20 15:54
I am not sure what to with the int constructor. Should it try __index__ before __int__? Or just make __index__ without __int__ setting the nb_int slot as was proposed by Nick in issue33039?
msg336127 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2019-02-20 17:14
> I am not sure what to with the int constructor. Should it try __index__ before __int__? Or just make __index__ without __int__ setting the nb_int slot as was proposed by Nick in issue33039?

Shouldn't it try only __int__ since this will default to __index__ once #33039 is closed?
msg336534 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-25 15:58
New changeset 6a44f6eef3d0958d88882347190b3e2d1222c2e9 by Serhiy Storchaka in branch 'master':
bpo-36048: Use __index__() instead of __int__() for implicit conversion if available. (GH-11952)
https://github.com/python/cpython/commit/6a44f6eef3d0958d88882347190b3e2d1222c2e9
msg340944 - (view) Author: Miro Hrončok (hroncok) * Date: 2019-04-26 21:39
Relevant NumPy issue: https://github.com/numpy/numpy/issues/13412
History
Date User Action Args
2019-06-03 10:20:47mark.dickinsonsetpull_requests: + pull_request13659
2019-04-26 21:39:39hroncoksetnosy: + hroncok
messages: + msg340944
2019-03-14 14:58:36serhiy.storchakalinkissue33002 superseder
2019-03-14 14:54:58serhiy.storchakalinkissue12974 superseder
2019-02-25 16:13:58serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-02-25 15:58:03serhiy.storchakasetmessages: + msg336534
2019-02-20 17:14:30remi.lapeyresetnosy: + remi.lapeyre
messages: + msg336127
2019-02-20 15:54:36serhiy.storchakasetmessages: + msg336107
2019-02-20 11:13:37vstinnersetmessages: + msg336071
2019-02-20 11:11:11vstinnersettitle: Deprecate implicit truncating when convert Python numbers to C integers -> Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__
2019-02-20 11:10:59vstinnersetmessages: + msg336069
2019-02-20 08:34:51serhiy.storchakasetmessages: + msg336050
2019-02-20 08:12:24serhiy.storchakasetnosy: + gvanrossum
2019-02-20 06:54:23serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request11977
2019-02-20 06:43:05serhiy.storchakacreate