classification
Title: array module: deprecate '__int__' conversion support for array elements
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__
View: 36048
Assigned To: Nosy List: Oren Milman, mark.dickinson, meador.inge, serhiy.storchaka
Priority: normal Keywords:

Created on 2011-09-14 00:48 by meador.inge, last changed 2019-03-14 14:54 by serhiy.storchaka. This issue is now closed.

Messages (7)
msg144000 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-09-14 00:48
When reviewing the fix for issue1172711 it was discovered that the 'array' module allows for '__int__' conversions:

>>> import array, struct
>>> a = array.array('L', [1,2,3])
>>> class T(object):
...     def __init__(self, value):
...         self.value = value
...     def __int__(self):
...          return self.value
...
>>> a = array.array('L', [1,2,3])
>>> struct.pack_into('L', a, 0, 9)
>>> a
array('L', [9, 2, 3])
>>> a[0] = T(100)
>>> a
array('L', [100, 2, 3])

As discussed in issue1172711, this behavior may not be desirable.  We should look at deprecating '__int__' and adding '__index__' as was done for the struct module in issue1530559.
msg144120 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-09-16 10:33
I just discovered that struct packs pointers from objects with an
__index__() method. Is that intentional?

>>> import struct
>>> class IDX(object):
...     def __init__(self, value):
...         self.value = value
...     def __index__(self):
...          return self.value
... 
>>> struct.pack('P', IDX(9))
b'\t\x00\x00\x00\x00\x00\x00\x00'
>>>
msg144126 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2011-09-16 11:59
Yes, that's intentional.  When use of __int__ was deprecated, a bug report popped up from someone who wanted to be able to have their own objects treated as integers for the purposes of struct.pack.  (I don't recall which issue;  Meador, do you remember?)  So we added use of __index__ at that point.

I think __index__ is the right interface for something to expose if it wants to be usable as an integer, and this usage is consistent with the original __index__ PEP.
msg144131 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-09-16 13:00
Mark Dickinson <report@bugs.python.org> wrote:
> Yes, that's intentional.  When use of __int__ was deprecated, a bug
> report popped up from someone who wanted to be able to have their own
> objects treated as integers for the purposes of struct.pack. 
> (I don't recall which issue;  Meador, do you remember?)
>  So we added use of __index__ at that point.

Yes, I think that's #1530559, and the bug report was about PyCUDA. I can
see why 'bBhHiIlLqQ' allow __index__(), since they previously allowed
__int__().

I specifically meant the 'P' format. As far as I can see, PyLong_AsVoidPtr()
never allowed __int__(), but now index objects can be packed as pointers.
It isn't a big deal, I just have to know for features/pep-3118.

To illustrate, this is python2.5.0; INT is an object with an __int__() method:

'\x07\x00\x00\x00\x00\x00\x00\x00'
>>> struct.pack('P', INT(7))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stefan/hg/r25/Lib/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: cannot convert argument to long
>>>
msg144135 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2011-09-16 15:03
> I specifically meant the 'P' format. As far as I can see, PyLong_AsVoidPtr()
> never allowed __int__(), but now index objects can be packed as pointers.
> It isn't a big deal, I just have to know for features/pep-3118.
>
> To illustrate, this is python2.5.0; INT is an object with an __int__() method:
>
> '\x07\x00\x00\x00\x00\x00\x00\x00'
>>>> struct.pack('P', INT(7))
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/home/stefan/hg/r25/Lib/struct.py", line 63, in pack
>    return o.pack(*args)
> struct.error: cannot convert argument to long

Huh, that's interesting.  It doesn't allow 'unsigned long' packs either (2.6.7):

Python 2.6.7+ (unknown, Sep 16 2011, 09:53:25)
[GCC 4.6.0 20110603 (Red Hat 4.6.0-10)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> class IDX(object):
...     def __init__(self, value):
...         self.value = value
...     def __int__(self):
...          return self.value
...
>>> for code in ['l', 'L', 'P']:
...    try:
...       struct.pack(code, IDX(9))
...    except Exception as e:
...       print "pack('%s'): %s" % (code, e)
...
'\t\x00\x00\x00'
pack('L'): unsupported operand type(s) for &: 'IDX' and 'long'
pack('P'): cannot convert argument to long

The behavior around '__int__' in previous versions seems somewhat accidental.
msg144138 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-09-16 16:56
Meador Inge <report@bugs.python.org> wrote:
> The behavior around '__int__' in previous versions seems somewhat accidental.

I think struct followed the functions in longobject.c, which is not
really consistent with respect to duck typing. See also #12965 or
http://bugs.python.org/issue1172711#msg48086.

But I think that the decision to accept __index__() for both signed
and unsigned integer formats is good for consistency.

For 'P' I'm not sure, but course it might be used in the wild by now.
msg336204 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-21 10:56
See more general issue36048.
History
Date User Action Args
2019-03-14 14:54:58serhiy.storchakasetstatus: open -> closed
superseder: Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__
resolution: duplicate
stage: needs patch -> resolved
2019-02-21 10:56:30serhiy.storchakasetmessages: + msg336204
2016-10-09 19:39:33Oren Milmansetnosy: + Oren Milman
2016-09-28 11:47:56serhiy.storchakasetnosy: + serhiy.storchaka
2014-10-14 14:42:14skrahsetnosy: - skrah
2011-09-16 16:56:05skrahsetmessages: + msg144138
2011-09-16 15:03:51meador.ingesetmessages: + msg144135
2011-09-16 13:00:04skrahsetmessages: + msg144131
2011-09-16 11:59:07mark.dickinsonsetmessages: + msg144126
2011-09-16 10:33:25skrahsetmessages: + msg144120
2011-09-14 00:48:15meador.ingecreate