classification
Title: errors raised by bytes() and bytearray() for invalid size argument
Type: enhancement Stage: test needed
Components: Interpreter Core Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Oren Milman, haypo, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-03-17 20:33 by Oren Milman, last changed 2017-03-24 20:47 by terry.reedy.

Messages (2)
msg289783 - (view) Author: Oren Milman (Oren Milman) * Date: 2017-03-17 20:33
currently (on my Windows 10):
>>> bytes(-1 << 1000)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer
>>> bytes(-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: negative count
>>> bytes(sys.maxsize + 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer

for the same size arguments, bytearray raises the same errors.

thus, in accordance with #29833 (this is a sub-issue of #29833) for each of the
constructors of bytes and bytearray:
    1. ValueErrors with the same error message should be raised for any
       negative size argument (big negative as well as small negative).
    2. MemoryError should be raised for any size argument bigger than
       sys.maxsize.


Moreover, currently:
>>> bytes(sys.maxsize - 25)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> bytes(sys.maxsize - 24)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: byte string is too large
>>> bytes(sys.maxsize)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: byte string is too large

for each of these size arguments, bytearray raises a MemoryError.

IMHO, to make the error messages more consistent, the constructor of bytes
should raise a MemoryError for any too large size argument, as the constructor
of bytearray already does.
msg289785 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-17 20:56
I worked on this issue. The simplest solution is calling PyNumber_AsSsize_t() with NULL rather than PyExc_OverflowError in bytes and bytearray constructors. Then both constructors will raise ValueError for large negative size and bytearray() will raise MemoryError for large positive size. For raising MemoryError in bytes() we should change OverflowError to MemoryError in other place.

But this is not the only difference between bytes and bytearray.

>>> bytearray(b'abcd') * sys.maxsize
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> b'abcd' * sys.maxsize
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: repeated bytes are too long

This looks related and I think that it is worth to change OverflowError to MemoryError in the repetition operation. But 'abcd' * sys.maxsize raises OverflowError too, therefore we should change exception types in str.

Concatenation also can raise OverflowError. If change OverflowError to MemoryError in above operations, it should be changed for concatenation too.
History
Date User Action Args
2017-03-24 20:47:05terry.reedysetstage: test needed
2017-03-24 20:46:50terry.reedysettitle: errors raised by bytes and bytearray constructors for invalid size argument -> errors raised by bytes() and bytearray() for invalid size argument
2017-03-17 20:56:36serhiy.storchakasetnosy: + haypo
messages: + msg289785
2017-03-17 20:33:15Oren Milmancreate