This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: += operator and accessors bug?
Type: behavior Stage: resolved
Components: Versions: Python 3.9
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: chovey, eric.smith, josh.r, mark.dickinson
Priority: normal Keywords:

Created on 2021-09-30 15:54 by chovey, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bug_plus_equals_numpy.py chovey, 2021-09-30 21:28
Messages (7)
msg402966 - (view) Author: (chovey) Date: 2021-09-30 15:54
We used get/set attribute accessors with private data, and suspect the beaviour we see with the += operator contains a bug.  Below is our original implementation, followed by our fix.  But, we feel the original implementation should have worked.  Please advise.  Thank you.

    @property
    def position(self) -> np.ndarray:
        return self._position

    @position.setter
    def position(self, val: np.ndarray):
        self._position = val

    def update(self, *, delta_t: float):
        # Nope! Bug! (?)  
        # Tried to combine the getter and setter for self.position
        # with the += operator, which will not work.
        # self.position += self.velocity * delta_t
        #
        # This fixes the behaviour.
        self._position = self.velocity * delta_t + self.position
msg402969 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-09-30 16:13
Please show what result you're getting, and what result you're expecting. "will not work" is not very helpful for us to replicate it.
msg402976 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-09-30 17:33
Did you by any chance get an error message resembling the following?

> "Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'"

(If you can give us a complete piece of code that we can run ourselves, that would save us from having to guess what the issue is.)
msg402984 - (view) Author: (chovey) Date: 2021-09-30 20:46
Let me get a minimum working example (MWE) developed and then I will return and paste it in here.  Ball in my court.  Than, you.
msg402988 - (view) Author: (chovey) Date: 2021-09-30 21:27
I confirm I do get this error:
Exception has occurred: _UFuncOutputCastingError
Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'

I next will paste a minimum working example.
msg402989 - (view) Author: (chovey) Date: 2021-09-30 21:28
import sys

import numpy as np


class Kinematics:
    def __init__(self, *, initial_position: np.ndarray, initial_velocity: np.ndarray):
        self._position = initial_position  # meters
        self._velocity = initial_velocity  # meters per second

    @property
    def position(self) -> np.ndarray:
        return self._position

    @position.setter
    def position(self, val: np.ndarray):
        self._position = val

    @property
    def velocity(self) -> np.ndarray:
        return self._velocity

    @velocity.setter
    def velocity(self, val: np.ndarray):
        self._velocity = val

    def update_shows_bug(self, *, delta_t: float):
        # Tries to combine the getter and setter for self.position
        # with the += operator, which will not work.
        # Will cause this error:
        # Exception has occurred: _UFuncOutputCastingError
        # Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'
        self.position += self.velocity * delta_t

    def update_fixes_bug(self, *, delta_t: float):
        # Fixes the bug exibited in the 'update_shows_bug' method.
        self._position = self.velocity * delta_t + self.position


def main(argv):

    # after an elapsed time of 2 seconds, calucate the new position
    dt = 2.0  # seconds, elapsed time step

    # construct a Kinematics object
    x0, y0 = 1000, 2000  # meters
    xdot0, ydot0 = 20, 30  # meters per second
    k1 = Kinematics(
        initial_position=np.array([x0, y0]), initial_velocity=np.array([xdot0, ydot0])
    )  # m and m/s
    k2 = Kinematics(
        initial_position=np.array([x0, y0]), initial_velocity=np.array([xdot0, ydot0])
    )  # m and m/s

    # expected updated position is rate * time + initial_position
    #
    # x-direction
    # = (20 m/s) * (2 s) + 1000 m
    # = 40 m + 1000 m
    # = 1040 m
    #
    # y-direction
    # = (30 m/s) * (2 s) + 2000 m
    # = 60 m + 2000 m
    # = 2060 m

    xf, yf = 1040, 2060  # meters

    # k1.update_shows_bug(delta_t=dt)  # will trigger error
    # new_position_with_bug = k1.position
    # assert new_position_with_bug[0] == xf  # meters, succeeds
    # assert new_position_with_bug[1] == yf  # meters, succeeds

    k2.update_fixes_bug(delta_t=dt)
    new_position_without_bug = k2.position
    assert new_position_without_bug[0] == xf  # meters, succeeds
    assert new_position_without_bug[1] == yf  # meters, succeeds

    print("Finished.")


if __name__ == "__main__":
    main(sys.argv[1:])
msg402991 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2021-09-30 23:48
This has nothing to do with properties, it's 100% about using augmented assignment with numpy arrays and mixed types. An equivalent reproducer is:

a = np.array([1,2,3])  # Implicitly of dtype np.int64

a += 0.5  # Throws the same error, no properties involved

The problem is that += is intended to operate in-place on mutable types, numpy arrays *are* mutable types (unlike normal integers in Python), you're trying to compute a result that can't be stored in a numpy array of integers, and numpy isn't willing to silently make augmented assignment with incompatible types make a new copy with a different dtype (they *could* do this, but it would lead to surprising behavior, like += on the *same* numpy array either operating in place or creating a new array with a different dtype and replacing the original based on the type on the right-hand side).

The short form is: If your numpy computation is intended to produce a new array with a different data type, you can't use augmented assignment. And this isn't a bug in CPython in any event; it's purely about the choices (reasonable ones IMO) numpy made implementing their __iadd__ overload.
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89496
2021-09-30 23:48:58josh.rsetstatus: open -> closed

nosy: + josh.r
messages: + msg402991

resolution: not a bug
stage: resolved
2021-09-30 21:28:15choveysetfiles: + bug_plus_equals_numpy.py

messages: + msg402989
2021-09-30 21:27:36choveysetmessages: + msg402988
2021-09-30 20:46:24choveysetmessages: + msg402984
2021-09-30 17:33:54mark.dickinsonsetnosy: + mark.dickinson
messages: + msg402976
2021-09-30 16:13:10eric.smithsetnosy: + eric.smith
messages: + msg402969
components: - 2to3 (2.x to 3.x conversion tool)
2021-09-30 15:54:52choveycreate