This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author grahamd
Recipients grahamd
Date 2013-09-22.10:35:42
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1379846143.45.0.326809671068.issue19070@psf.upfronthosting.co.za>
In-reply-to
Content
When a weakref.proxy() is used to wrap a class instance which implements in place operators, when one applies the in place operator to the proxy, one could argue the variable holding the proxy should still be a reference to the proxy after the in place operator has been done. Instead the variable is replaced with the class instance the proxy was wrapping.

So for the code:

from __future__ import print_function

import weakref

class Class(object):
     def __init__(self, value):
         self.value = value
     def __iadd__(self, value):
         self.value += value
         return self

c = Class(1)

p = weakref.proxy(c)

print('p.value', p.value)
print('type(p)', type(p))

p += 1

print('p.value', p.value)
print('type(p)', type(p))

one gets:

$ python3.3 weakproxytest.py
p.value 1
type(p) <class 'weakproxy'>
p.value 2
type(p) <class '__main__.Class'>

One might expect type(p) at the end to still be <class 'weakproxy'>.

In the weakref.proxy() C code, all the operators are set up with preprocessor macros.

#define WRAP_BINARY(method, generic) \
    static PyObject * \
    method(PyObject *x, PyObject *y) { \
        UNWRAP(x); \
        UNWRAP(y); \
        return generic(x, y); \
    }

#define WRAP_TERNARY(method, generic) \
    static PyObject * \
    method(PyObject *proxy, PyObject *v, PyObject *w) { \
        UNWRAP(proxy); \
        UNWRAP(v); \
        if (w != NULL) \
            UNWRAP(w); \
        return generic(proxy, v, w); \
    }

These are fine for:

WRAP_BINARY(proxy_add, PyNumber_Add)
WRAP_BINARY(proxy_sub, PyNumber_Subtract)
WRAP_BINARY(proxy_mul, PyNumber_Multiply)
WRAP_BINARY(proxy_div, PyNumber_Divide)
WRAP_BINARY(proxy_floor_div, PyNumber_FloorDivide)
WRAP_BINARY(proxy_true_div, PyNumber_TrueDivide)
WRAP_BINARY(proxy_mod, PyNumber_Remainder)
WRAP_BINARY(proxy_divmod, PyNumber_Divmod)
WRAP_TERNARY(proxy_pow, PyNumber_Power)
WRAP_BINARY(proxy_lshift, PyNumber_Lshift)
WRAP_BINARY(proxy_rshift, PyNumber_Rshift)
WRAP_BINARY(proxy_and, PyNumber_And)
WRAP_BINARY(proxy_xor, PyNumber_Xor)
WRAP_BINARY(proxy_or, PyNumber_Or)

Because a result is being returned and the original is not modified.

Use of those macros gives the unexpected result for:

WRAP_BINARY(proxy_iadd, PyNumber_InPlaceAdd)
WRAP_BINARY(proxy_isub, PyNumber_InPlaceSubtract)
WRAP_BINARY(proxy_imul, PyNumber_InPlaceMultiply)
WRAP_BINARY(proxy_idiv, PyNumber_InPlaceDivide)
WRAP_BINARY(proxy_ifloor_div, PyNumber_InPlaceFloorDivide)
WRAP_BINARY(proxy_itrue_div, PyNumber_InPlaceTrueDivide)
WRAP_BINARY(proxy_imod, PyNumber_InPlaceRemainder)
WRAP_TERNARY(proxy_ipow, PyNumber_InPlacePower)
WRAP_BINARY(proxy_ilshift, PyNumber_InPlaceLshift)
WRAP_BINARY(proxy_irshift, PyNumber_InPlaceRshift)
WRAP_BINARY(proxy_iand, PyNumber_InPlaceAnd)
WRAP_BINARY(proxy_ixor, PyNumber_InPlaceXor)
WRAP_BINARY(proxy_ior, PyNumber_InPlaceOr)

This is because the macro returns the result from the API call, such as PyNumber_InPlaceAdd() where as it should notionally be returning 'proxy' so that the variable holding the weakref proxy instance is set to the proxy object again and not the result of the inner API call.

In changing this though there is a complication which one would have to deal with.

If the result of the inner API call such as PyNumber_InPlaceAdd() is the same as the original object wrapped by the weakref proxy, then all is fine.

What though should be done if it is different as notionally this means that the reference to the wrapped object would need to be changed to the new value.

The issue is that if one had to replace the reference to the wrapped object with a different one due to the in place operator, then notionally the whole existence of that weakref is invalidated as you would no longer be tracking the same object the weakref proxy was created for.

This odd situation is perhaps why the code was originally written the way it was, although that then sees the weakref proxy being replaced which could cause different problems with the callback not later being called since the weakref proxy can be destroyed before the object it wrapped. As there is nothing in the documentation of the code which calls out such a decision, not sure if it was deliberate or simply an oversight.

Overall I am not completely sure what the answer should be, so I am logging it as interesting behaviour. Maybe this odd case needs to be called out in the documentation in some way at least. That or in place operators simply shouldn't be allowed on a weakref proxy because of the issues it can cause either way.
History
Date User Action Args
2013-09-22 10:35:43grahamdsetrecipients: + grahamd
2013-09-22 10:35:43grahamdsetmessageid: <1379846143.45.0.326809671068.issue19070@psf.upfronthosting.co.za>
2013-09-22 10:35:43grahamdlinkissue19070 messages
2013-09-22 10:35:42grahamdcreate