This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add typing.py class describing a PEP 3118 buffer object
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Daniel Moisset, gvanrossum, jstasiak, levkivskyi, rhettinger
Priority: normal Keywords: patch

Created on 2016-07-12 18:07 by Daniel Moisset, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
buffer.patch Daniel Moisset, 2016-07-14 14:16 review
Messages (8)
msg270261 - (view) Author: Daniel Moisset (Daniel Moisset) * Date: 2016-07-12 18:07
The buffer protocol is a low level C protocol, but even if it doesn't expose a specific python API, it's currently not possible to say "this object implements the buffer protocol" in Python

This might be useful for some applications, including PEP-484 typing hints. It would be useful to have a collections.abc.Buffer which already registers the types that already support the protocol (bytes, strings, array.array, struct, mmap, ctype arrays/pointers, etc)
msg270315 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-07-13 15:22
I'll defer to typing gurus as to whether something like this might make sense in typing.py, but it doesn't make much sense as a collections.abc without an API exposed in pure python and without useful mixin methods.  By way of comparison, consider that the named tuple protocol is in typing.NamedTuple but  not in collections.abc; instead, we just use _fields to recognize it when the need arises (which is almost never).   

I would like collections.abc to remain a place for ABCs that can be usefully subclassed, that provide guaranteed behaviors, and that are in fact related to collections.  The module's utility will be impaired if it becomes a dumping ground for miscellaneous type hinting concepts and registrations.

Early on there was some experimentation with the "one-trick ponies" in collections.abc such as Callable and Hashable that turned-out to almost never be used in practice, so we don't want to continue to that line of development when there are more promising avenues such as typing.py.
msg270321 - (view) Author: Daniel Moisset (Daniel Moisset) * Date: 2016-07-13 16:13
Thanks for the feedback. Just for the context, I opened this issue also based on this email: http://permalink.gmane.org/gmane.comp.python.devel/156411

 Probably there are two different discussions to have here:

1) Is the "Buffer" a useful ABC to have even if there's no python API involved? My thoughts (that come from working with typing and PEP 484 and numpy) are "yes"

2) Where is the right place to put it? I suggested collections.abc after seeing the thread I linked, although I wouldn't mind if this was in typing.py or somewhere else.
msg270323 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-07-13 16:38
Since collections.abc is the wrong place, I'll change the title of this to typing and see if any of the typing aficionados think it is sufficiently useful (they are deliberately focusing on common use cases first since the whole static typing endeavor is exploratory and this limits of what should be done and what isn't really needed or useful isn't yet well understood).
msg270334 - (view) Author: Daniel Moisset (Daniel Moisset) * Date: 2016-07-13 18:22
OK. Just to give an obvious example of a place where this would be useful for annotations is in the argument for the memoryview() builtin (currently declared with "# TODO arg can be any obj supporting the buffer protocol" in the standard library typesheds)
msg270412 - (view) Author: Daniel Moisset (Daniel Moisset) * Date: 2016-07-14 14:16
As a description of the idea I'm attaching a rough patch with an implementation. Some tests fail because of the following:

1) I didn't find a good place to register the Buffer ABC for mmap.mmap, and array.mmap; the modeules themselves are C extensions (and it's possible but weird to register the ABC from the C API); I could import those in typing.py but then I'd make typing depend from mmap and array which also feels wrong; so I didn't do that and the related tests that I added fail.

2) I had a similar situation with io.BytesIO. In that case there's a python module and I'm doing the abc.register there, but now io imports typing, and the test_site fails because many new modules (typing and its dependencies) are imported on startup. The alternative is to make typing depend on io (in this case it could be a reasonable idea)

After these, I'm feeling that typing.py may not be the best place for this (but given that collections.abc isn't either, I'm not sure where's the right place)
msg270437 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-07-14 20:47
Looking at the patch, I'm thinking that this endeavor isn't at all worthwhile.  We don't need an ABC for everything especially if there in no pure python API involved.  I recommend closing this one until there is a proven need and a clear path forward.
msg357522 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2019-11-26 18:30
Cross-ref to the typing issue https://github.com/python/typing/issues/593. It looks like there is some interest in this feature.
History
Date User Action Args
2022-04-11 14:58:33adminsetgithub: 71688
2020-05-19 18:11:43gvanrossumsetnosy: + gvanrossum
2020-05-19 07:38:17jstasiaksetnosy: + jstasiak
2019-11-26 18:30:42levkivskyisetmessages: + msg357522
2016-07-14 20:47:15rhettingersetmessages: + msg270437
2016-07-14 14:16:06Daniel Moissetsetfiles: + buffer.patch
keywords: + patch
messages: + msg270412
2016-07-14 06:34:48levkivskyisetnosy: + levkivskyi
2016-07-13 18:22:50Daniel Moissetsetmessages: + msg270334
2016-07-13 16:38:58rhettingersetmessages: + msg270323
title: Create a collections.abc class that describes PEP 3118 buffer -> Add typing.py class describing a PEP 3118 buffer object
2016-07-13 16:13:13Daniel Moissetsetmessages: + msg270321
2016-07-13 15:22:44rhettingersetnosy: + rhettinger

messages: + msg270315
versions: + Python 3.6
2016-07-12 18:09:55Daniel Moissetsettype: enhancement
components: + Library (Lib)
2016-07-12 18:07:23Daniel Moissetcreate