New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Regression in bytes constructor #73345
Comments
Consider the following code: $ cat bug.py
from array import array
class C(array):
def __new__(cls):
return array.__new__(cls, 'B', b'abc')
def __index__(self):
raise TypeError
x = C()
print(bytes(x)) It works under python 3.5: $ python3.5 bug.py
b'abc' but raises a TypeError under python 3.6: $ python3.6 bug.py
Traceback (most recent call last):
File "bug.py", line 8, in <module>
print(bytes(x))
File "bug.py", line 6, in __index__
raise TypeError
TypeError It looks like this was introduced in issue bpo-27704. (Ref: e/pyq#827) |
My test code may seem contrived, but numpy arrays exhibit similar behavior: >>> a = numpy.array([2, 2])
>>> bytes(a)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: only integer arrays with one element can be converted to an index Under python 3.5, the result was >>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00' |
Indeed, there is behavior change. It is easy to revert the old behavior. But What is a reason of making an array an integer type? |
Ah, if numpy arrays are affected, this should be restored. But swallowing an arbitrary exception doesn't look good to me. |
I don't think we should put too much effort into preserving numpy behavior. Consider this python 3.5 session: >>> import numpy
>>> a = numpy.array([1])
>>> bytes(a)
__main__:1: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
b'\x00'
>>> a = numpy.array([2, 2])
>>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00' It looks like this behavior is just an artifact of ndarray providing both __index__ and buffer protocol and not something thought out by numpy developers. I wonder if we could check for buffer protocol support before detecting an integer argument? I also recall a discussion of deprecating bytes(int) altogether. See <https://mail.python.org/pipermail/python-ideas/2014-March/027295.html\>. |
Inada, Can you explain what your patch does? I don't understand why you single out the OverflowError. When is __index__ expected to raise it? >>> (2**300).__index__()
2037035976334486086268445688409378161051468393665936250636140449354381299763336706183397376 |
@belopolsky See code review discussion in bpo-27704. |
@inada.naoki Sorry, I still don't understand the role of OverflowError. With respect to my earlier suggestion that buffer protocol should have priority over __index__, note that the documentation implies the same order:
class bytes(object)
| bytes(iterable_of_ints) -> bytes
| bytes(string, encoding[, errors]) -> bytes
| bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer
| bytes(int) -> bytes object of size given by the parameter initialized with null bytes
| bytes() -> empty bytes object
|
| Construct an immutable array of bytes from:
| - an iterable yielding integers in range(256)
| - a text string encoded using the specified encoding
| - any object implementing the buffer API.
| - an integer
.. |
On the other hand, the documentation for the bytearray constructor 1 lists integer argument first. |
The code restored by 29159-index-fallback.patch is slightly different from the code before bpo-27704, but the patch LGTM except that arguments of assertEqual are in reversed order.
PyNumber_AsSsize_t(arg, PyExc_OverflowError) raises an OverflowError when the result of __index__ don't fit in Py_ssize_t.
This would fix an issue with NumPy arrays, but it is much harder to implement. |
New changeset 505cc50ddc82 by INADA Naoki in branch '3.6': |
fixed. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: