This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Caching/interning of small ints sometimes fails
Type: behavior Stage: patch review
Components: Interpreter Core Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: brandtbucher Nosy List: Dennis Sweeney, brandtbucher, steven.daprano
Priority: normal Keywords: patch

Created on 2022-03-08 15:48 by steven.daprano, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31843 open Dennis Sweeney, 2022-03-13 08:08
Messages (4)
msg414762 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2022-03-08 15:48
I'm reluctant to call this a bug, as small int interning/caching is an implementation detail and there are no hard guarantees made.

But on the other hand, it seems that the intention is that small ints such as 0, 1 and 2 should be cached. Here are some examples where they are not. Intentional or a bug?

>>> x = 1
>>> y = pow(2, 31, 2**31-1)
>>> y == x
True
>>> y is x
False


>>> x = 2
>>> y = pow(2, 31, 2**31-2)
>>> y == x
True
>>> y is x
False


It also affects values which are presumably constant-folded at compile time:

>>> x = 1
>>> y = 2**31 % (2**31 - 1)
>>> z = 2**31 % (2**31 - 1)
>>> x == y == z
True
>>> x is y
False
>>> y is z
False
>>> x is z
False



But if you run the code in exec, the value is interned:

>>> code = """
... x = 1
... y = 2**31 % (2**31-1)
... """
>>> dis(code)
  2           0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (x)

  3           4 LOAD_CONST               0 (1)
              6 STORE_NAME               1 (y)
              8 LOAD_CONST               1 (None)
             10 RETURN_VALUE
>>> exec(code)
>>> x is y
True



Also affects zero:

>>> x = 0
>>> y = 2**29 % (2**29)
>>> x is y
True
>>> y = 2**30 % (2**30)
>>> x is y
False


First noted here:

https://discuss.python.org/t/cached-integer-id-on-high-calculations/14128/1


>>> sys.version
'3.10.0 (default, Oct 28 2021, 20:43:43) [GCC 8.3.1 20190223 (Red Hat 8.3.1-2)]'
msg414764 - (view) Author: Alex Waygood (AlexWaygood) * (Python triager) Date: 2022-03-08 17:01
I think this might be a duplicate of Issue46361?
msg414766 - (view) Author: Brandt Bucher (brandtbucher) * (Python committer) Date: 2022-03-08 18:24
Related, except this seems to be happening in long_pow. I’ll take a look at it today.
msg415030 - (view) Author: Dennis Sweeney (Dennis Sweeney) * (Python committer) Date: 2022-03-13 07:48
I believe the issue is the usage of the x_divrem function.

x_divrem always returns fresh ints, never cached small ints. This behavior is relied upon in the long_true_divide function, as it mutates the returned quotient at the line """x->ob_digit[0] = low & ~(2U*mask-1U);""".

The other uses of x_divrem account for this when handling the *quotient*, but apparently missed checking for small ints in the *remainder*.

uses of x_divrem:
    - long_divrem
        - uses maybe_small_long on quotient
        - doesn't check if remainder is small <---- oops
    - long_rem
        - throws away quotient
        - doesn't check if remainder is small <---- oops
    - long_true_divide
        - modifies the quotient
        - throws away remainder


Possible patches to fix it:

1) Modify long_divrem and long_rem to check for small remainders, in addition to the small-quotient checks they already do.
2) Modify x_divrem to check for small remainders, but still always return fresh quotients.
3) Modify x_divrem to return cached quotients and remainders, and modify long_true_divide to make a copy before mutating.

I'd lean towards #1, since that quotient check already exists.
In #2, the mismatch of checking/not checking between quotient/remainder would be confusing.
In #3, an extra int allocation gets added to integer true divide, which isn't ideal.
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91117
2022-03-13 09:38:02AlexWaygoodsetnosy: - AlexWaygood
2022-03-13 08:08:05Dennis Sweeneysetkeywords: + patch
stage: patch review
pull_requests: + pull_request29942
2022-03-13 07:48:12Dennis Sweeneysetnosy: + Dennis Sweeney
messages: + msg415030
2022-03-08 18:24:11brandtbuchersetassignee: brandtbucher
messages: + msg414766
2022-03-08 18:21:06brandtbuchersetnosy: + brandtbucher
2022-03-08 17:01:19AlexWaygoodsetnosy: + AlexWaygood
messages: + msg414764
2022-03-08 15:48:45steven.dapranocreate