This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add unified C API for accessing bytes and bytearray
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: matrixise, ronaldoussoren, salty-horse, serhiy.storchaka
Priority: normal Keywords:

Created on 2019-02-21 14:20 by salty-horse, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (7)
msg336218 - (view) Author: Ori Avtalion (salty-horse) * Date: 2019-02-21 14:20
It would be useful to have a shared API for consuming bytes and bytearrays.

At present, I need to write very similar code twice. Some existing codebases only support bytes (perhaps forgetting bytearrays exist). Adding support for bytearray would be trivial if there was a shared API.

These are the functions/macros that can have "BytesOrByteArray" equivalents:
* PyBytes_Check
* PyBytes_CheckExact
* PyBytes_Size
* PyBytes_GET_SIZE
* PyBytes_AsString
* PyBytes_AS_STRING
* PyBytes_AsStringAndSize

Here are some example implementations for the macros:

#define PyBytesOrByteArray_Check(ob) (PyBytes_Check(ob) || PyByteArray_Check(ob))
#define PyBytesOrByteArray_AS_STRING(ob) (PyBytes_Check(ob) ? PyBytes_AS_STRING(ob) : PyByteArray_AS_STRING(ob))
#define PyBytesOrByteArray_GET_SIZE(ob) #define PyByteArray_GET_SIZE(self) (assert(PyBytesOrByteArray_Check(self)), Py_SIZE(self))
msg336219 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-02-21 14:21
I remove 2.7 because this branch is in bugfix mode, no new features.
msg336220 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-02-21 14:23
Stéphane: Please don't add me to the nosy list of bugs.
msg336222 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2019-02-21 14:30
What is your use case for this? Is that something that can use the buffer API instead of these low-level APIs?
msg336227 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-21 14:53
The unified C API already exists. It is called the buffer protocol.

https://docs.python.org/3/c-api/buffer.html#buffer-related-functions
msg336232 - (view) Author: Ori Avtalion (salty-horse) * Date: 2019-02-21 15:44
My use-case is modifying existing code that supports bytes to also support bytearray.

https://github.com/mongodb/mongo-python-driver/blob/9902d239b4e557c2a657e8c8110f7751864cec95/bson/_cbsonmodule.c#L1112

The buffer protocol, which I didn't know of, feels slightly complicated for my use-case. For now I opted to adding these macros myself, only changing 3 lines.
msg336233 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-02-21 16:01
If you need to support only bytes and bytearray, but not other bytes-like object, this is a too special case. It is easy to write your own macros or functions that wrap existing C API. Other option -- duplicate the code and replace PyBytes_ with PyByteArray_. In all cases be aware abot differences between bytes and bytearray: bytarray can change its content and size, saved values of PyByteArray_AS_STRING(ob) and PyByteArray_GET_SIZE(self) can not be used after executing arbitrary code in destructors or releasing GIT.
History
Date User Action Args
2022-04-11 14:59:11adminsetgithub: 80246
2019-02-21 16:01:51serhiy.storchakasetstatus: open -> closed
resolution: rejected
messages: + msg336233

stage: resolved
2019-02-21 15:44:11salty-horsesetmessages: + msg336232
2019-02-21 14:53:16serhiy.storchakasetmessages: + msg336227
2019-02-21 14:30:43ronaldoussorensetnosy: + ronaldoussoren
messages: + msg336222
2019-02-21 14:23:05vstinnersetnosy: - vstinner
2019-02-21 14:23:01vstinnersetnosy: salty-horse, vstinner, serhiy.storchaka, matrixise
messages: + msg336220
2019-02-21 14:21:26matrixisesetnosy: + serhiy.storchaka, matrixise, vstinner

messages: + msg336219
versions: - Python 2.7
2019-02-21 14:20:08salty-horsecreate