classification
Title: PyBytes_FromObject accepts arbitrary iterable
Type: behavior Stage:
Components: Versions: Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ZackerySpytz, martin.panter, pitrou, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2016-04-14 18:13 by serhiy.storchaka, last changed 2018-09-15 16:32 by ZackerySpytz.

Messages (4)
msg263423 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-04-14 18:13
PyBytes_FromObject creates a bytes object from an object that implements the buffer or the iterable protocol. But only using the buffer protocol is documented.

We should either document the current behavior (the documentation of int.from_bytes() can be used as a sample), or change the behavior to match the documentation.

For now PyBytes_FromObject() is used in the stdlib only for converting FS paths to str (besides using internally in bytes). When called from PyUnicode_FSDecoder(), this leads to accepting arbitrary iterables as filenames, that looks at leas strange (issue26754). In the posix module it is called only for objects that support the buffer protocol.

Thus the support of the iterable protocol is not used or misused in the stdlib. I don't know if it is used correctly in third party code, I suspect that this is rather misused. Note that there is alternative API function PyObject_Bytes(), that accepts same arguments as the bytes() constructor, except an integer, and supports the buffer protocol, the iterable protocol, and in additional supports the __bytes__() special method.
msg263425 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2016-04-14 18:29
> Note that there is alternative API function PyObject_Bytes(), that accepts same arguments as the bytes() constructor, except an integer, and supports the buffer protocol, the iterable protocol, and in additional supports the __bytes__() special method.

That's a good point. Then I think PyBytes_FromObject() should be restricted. Also the docs for these functions should probably mention each other.
msg263426 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-04-14 18:45
Other *_FromObject public API functions:

PyUnicode_FromObject() -- Very strict. Accepts only str subclasses.
PyByteArray_FromObject() -- Very lenient! Calls the bytearray() constructor, accepts even an integer!
PyMemoryView_FromObject() -- No surprises. Accepts only objects that support the buffer protocol.
msg325445 - (view) Author: Zackery Spytz (ZackerySpytz) * (Python triager) Date: 2018-09-15 16:32
I've created #34696 for PyByteArray_FromObject().
History
Date User Action Args
2018-09-15 16:32:04ZackerySpytzsetnosy: + ZackerySpytz
messages: + msg325445
2016-04-14 18:45:51serhiy.storchakasetmessages: + msg263426
2016-04-14 18:29:35pitrousetmessages: + msg263425
2016-04-14 18:13:43serhiy.storchakacreate