Author jonathanunderwood
Recipients jonathanunderwood, r.david.murray, serhiy.storchaka
Date 2017-12-27.18:37:02
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1514399822.52.0.213398074469.issue32431@psf.upfronthosting.co.za>
In-reply-to
Content
Py_SIZE is actually precisely specified and documented[1] as it stands; I don't think a change there is needed. The usage I outlined is in line with that documentation, and many other uses of that macro in the cpython sources.

The documentation issues are, at least:

1. There is no documentation specifying that bytes objects should be null terminated.

2. Nothing in the documentation of PyBytes_FromStringAndSize[2] specifies that passing 0 as the size results in a singleton being returned. This is undocumented behaviour, and it would seem fragile to rely on this.

But there are more implementation inconsistencies: the documentation for PyBytes_AsString()[3] returns a buffer which is one byte longer than the length of the object *in order to store a terminating null*, which implies that the object need not itself have a terminating null. I could go on with other examples, but this is very poorly defined behaviour.

Question: are bytes objects defined to be null terminated, or not?

Because if they're not defined to be null terminated, the fix I propose is correct even if it doesn't solve the other 100 bugs lurking in the code.

[Aside: even if bytes objects are in fact defined to be null terminated, I think the change proposed amounts to an optimization in any case.]

[1] https://docs.python.org/3/c-api/structures.html#c.Py_SIZE
[2] https://docs.python.org/3/c-api/bytes.html?highlight=pybytes_fromstringandsize#c.PyBytes_FromStringAndSize
[3] https://docs.python.org/3/c-api/bytes.html?highlight=pybytes_asstring#c.PyBytes_AsString
History
Date User Action Args
2017-12-27 18:37:02jonathanunderwoodsetrecipients: + jonathanunderwood, r.david.murray, serhiy.storchaka
2017-12-27 18:37:02jonathanunderwoodsetmessageid: <1514399822.52.0.213398074469.issue32431@psf.upfronthosting.co.za>
2017-12-27 18:37:02jonathanunderwoodlinkissue32431 messages
2017-12-27 18:37:02jonathanunderwoodcreate