New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
collections.abc.Indexable #70176
Comments
In an -ideas thread, Guido suggested (http://article.gmane.org/gmane.comp.python.ideas/37599):
Later in the thread, Nick Coghlan suggested (http://article.gmane.org/gmane.comp.python.ideas/37614):
So, in particular:
This means that you can write this (borrowing an example from Steven D'Aprano in http://article.gmane.org/gmane.comp.python.ideas/23369/): class Squares(collections.abc.Indexable):
def __getitem__(self, index):
return index**2 Because this no longer depends on the old-style sequence protocol, testing it with ABCs will work as expected. For related issues, see bpo-25987, bpo-25864, bpo-25958, and python/typing#170 |
Interesting. I just hope the name doesn't confuse people into connecting this with the Index type (a type of integer that can't be cast from float). |
The last few weeks something bothered while working on Protocols PEP: protocols should be ideally compact (and PEP already emphasizes this). Fortunately I stumbled into this issue. It looks like the optimal way now is:
Guido, Łukasz if you agree, then I will add this to the Protocols PEP and will make a PR to collections.abc. We need to agree on the name, there are two options now: BaseMap and Indexable. I don't like the second since I would rather have it as a generic alias: Indexable = BaseMap[int, T]. However, BaseMap is also not very good, since I want something more "neutral" between Mapping and Sequence. |
IIRC, the original motivation for ABCs was to differentiate distinct uses of __getitem__ (we forever struggled with differentiating sequences from mapping). It seems to me that this proposal is a step backwards. Other than a feeling of lightness, I don't think this proposal does anything for us. What is point of knowing an object is Subscriptable without knowing how it is to be used. The OP has a sense that Mapping and Sequence are "too heavy" but I think the reality that useful classes almost never use __getitem__ in isolation; rather, it is part of a small constellation of methods that are typically used together. I would prefer that collections.abc continue to reflect that reality. Also, I worry that collections.abc is becoming cluttered. The existence of use ABCs like MutableMapping is being drowned-out by one-trick-ponies. We're developing an unfavorable ratio of theoretical building blocks versus the practical tools. |
Raymond,
Why do you think they are "theoretical"? FWIW, a simple search on GitHub for abc.X gives this: I don't see any pattern here (although this may be only anecdotal evidence :-)
This proposal makes more sense (and motivation) in the context of static typing, since it is a good way to know how __getitem__ is going to be used -- from its static type. Something typed with Subscriptable[int, T] would accept both Sequence[T] and Mapping[int, T]. The latter two are currently differentiated by __reversed__ (Sequence is Reversible as opposed to Mapping). Concerning an invariant that order of iteration is consistent with indexing by subsequent integers, I think this can't be checked reliably neither statically, nor by any simple runtime isinstance check. For example a subclass can override Indexable.__iter__ and break the iteration order. We can still add it, and rely on user cooperation. Anyway, I am not insisting on adding either Subscriptable nor Indexable, but I think it is important that these things are discussed in view of PEP-544. Alternative proposal would be to add Subscriptable and its subclass Indexable only to typing as protocols (with some special-casing in type checkers for the latter to provide a higher level of contract). |
At least one use of Indexable (by whatever name, but this rolls off the tongue the easiest) would be the metaclass in typing.py that allows one to write List[int] -- there's no containter here! The "drowning out" seems purely subjective -- perhaps if you order the contents of the module alphabetically you won't get much insight, but the docs should use a better organizing principle than that. (After all alphabetization was invented as a crutch for searching. :-) |
Guido, do you think there is some way to decouple collections.abc from spilling into the namespace for non-abc collections module? At the interactive prompt, running dir() on collections gives an alphabetical hodgepodge of the two modules. To my eyes, it is difficult to discern the concrete from the abstract. >>> dir(collections)
['AsyncGenerator', 'AsyncIterable', 'AsyncIterator', 'Awaitable', 'ByteString', 'Callable', 'ChainMap', 'Collection', 'Container', 'Coroutine', 'Counter', 'Generator', 'Hashable', 'ItemsView', 'Iterable', 'Iterator', 'KeysView', 'Mapping', 'MappingView', 'MutableMapping', 'MutableSequence', 'MutableSet', 'OrderedDict', 'Reversible', 'Sequence', 'Set', 'Sized', 'UserDict', 'UserList', 'UserString', 'ValuesView', '_Link', '_OrderedDictItemsView', '_OrderedDictKeysView', '_OrderedDictValuesView', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_chain', '_class_template', '_collections_abc', '_count_elements', '_eq', '_field_template', '_heapq', '_iskeyword', '_itemgetter', '_proxy', '_recursive_repr', '_repeat', '_repr_template', '_starmap', '_sys', 'abc', 'defaultdict', 'deque', 'namedtuple'] |
Names from collections.abc are re-exported to collections for backward compatibility. IIRC Serhiy also wanted to stop re-exporting them. I am not sure whether we need any "deprecation period" for this. |
I'm not sure that *new* names in collections.abc should be re-exported to collections. This isn't required for backward compatibility. |
Yes we will need a deprecation period for this starting with 3.7. It's fine |
There are two ways to do this. A wimpy and clean way is to just add a deprecation notice in the docs. The thorough and messy way is to temporarily copy everything from the 3.6 collections.abc into the regular collections module and then insert deprecation warning code directly in each of copied classes. |
The copying approach likely won't work because there will be code that uses isinstance(x, collections.abc.Sequence) and other code that uses isinstance(x, collections.Sequence) and they'll expect both to be true. |
If we want to add programming warnings, we should replace sys.modules['collections'] with an instance of a module type subclass with corresponding properties raising a DeprecationWarning. |
Then it seems like just adding a note to the docs is the way to go. That's probably okay though. I don't think we've every gotten much benefit from the programmatic deprecation warnings. |
I think it's not worth the hack that Serhiy suggests. But we should make |
Guido:
How about a paragraph in the Deprecation section of the "What's New in Python 3.7"? document |
You mean other than what's there? The text that's currently there begins with "In Python 3.8, ..." which almost sounds like it's misplaced or a typo. Maybe you can amend it somehow to explain that in 3.7, using or importing the ABCs from "collections" instead of from "collections.abc" is deprecated, and in 3.8 it will stop working. |
Can't we use the new shiny PEP-562 here to actually emit a warning when collections.Iterable is used instead of collections.abc.Iterable? |
PR 5460 implements this. It exposes issues in third-party packages. urllib3 and pyparsing directly use or import ABCs from 'collections'. request and pip are indirectly affected. If not add these warnings in 3.7 we can break the world in 3.8. |
Good idea, Victor, and thanks for the PR, Serhiy! |
The last commit that added the deprecation warning needs to be added to the 3.7 branch. |
Thanks, Tim, for noticing that and thanks, Ivan, for taking care of it. I should have cherrypicked this into 3.7.0b1. |
Thank you for all the hours all of you spent on making Python awesome. I would like to start contributing as well. Do you consider it time to remove |
mbussonn: Your new PR looks like it's related to bpo-36953 ("Remove collections ABCs?"), not this issue specifically. Can you withdraw/reissue attached to the correct issue? |
mbussonn: Your new PR looks like it's related to bpo-36953 ("Remove collections ABCs?"), not this issue specifically. Can you withdraw/reissue attached to the correct issue? Apologies, I've rebased and updated the issue numbers, not sure how I got the wrong one. I've also unlinked from here. |
…ast Python 3.6 When depending on this library, it's currently not possible to upgrade past Python 3.6 as KeysView, ValuesView and many other things are no longer exposed via `collections` but via `collections.abc` (see: python/cpython#70176). This change is backwards compatible and is tested to work from Python 3.6.* to Python 3.10.* at least.
…grade (#1) * Import KeysView, ValuesView from collections.abc to unblock upgrade past Python 3.6 When depending on this library, it's currently not possible to upgrade past Python 3.6 as KeysView, ValuesView and many other things are no longer exposed via `collections` but via `collections.abc` (see: python/cpython#70176). This change is backwards compatible and is tested to work from Python 3.6.* to Python 3.10.* at least. * Update version and changelog Co-authored-by: Daniel Pepper <pepper.daniel@gmail.com>
Importing abstract base classes has been marked as deprecated in Python 3.3[0] and the deprecation has been finished in Python 3.9[1] [0] python/cpython#55294 [1] python/cpython#70176
Importing abstract base classes from collections has been marked as deprecated in Python 3.3[0] in favor of importing from collections.abc. The deprecation has been finished in Python 3.9[1] [0] python/cpython#55294 [1] python/cpython#70176
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: