classification
Title: Move non-collections-related ABCs out of collections.abc
Type: enhancement Stage: test needed
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, ezio.melotti, gvanrossum, martin.panter, ncoghlan, rhettinger, stutzbach, terry.reedy, yselivanov
Priority: normal Keywords:

Created on 2015-11-16 19:21 by brett.cannon, last changed 2016-01-05 21:14 by brett.cannon. This issue is now closed.

Messages (14)
msg254744 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-11-16 19:21
In discussing adding a context manager ABC in issue #25609, the idea of creating a new module for ABCs that is more analogous to the types modules came up. Currently all ABCs that are generic in the stdlib and don't belong to a specific module end up in collections.abc, e.g. the Awaitable ABC has nothing to do with data structures and yet it's in a module in the collections package.

What we could do is create an interfaces module which contains all the ABCs that are not specific to a package. This would mean moving the ABCs out of collections.abc that have nothing to do with collections such as things for async, etc. The real question, though, is how to decide what belongs where, e.g. is Iterable/Iterator a collection, and if so then what about Generator?

The other option is that instead of introducing a generic interfaces module we simply create appropriate new modules to house the thematic ABCs and avoid a dumping ground scenario, e.g. some new module for the async stuff instead of just dumping them in an interfaces module. All the other ABCs could probably find existing homes, e.g. Callable could go into functools (although do we want to start a practice of having an abc submodule for all instances where ABCs exist and thus introduce functools.abc?).
msg254747 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-11-16 19:59
We had this discussion when we were adding Awaitable, AsyncIterable etc.  I suggested to simply put them in the top-level 'abc' module (where ABCMeta is defined).
msg254750 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-11-16 20:12
I guess is the objection of putting concrete ABCs in the abc module is that the abc module contains the actual code to implement ABCs and not ABCs themselves. Personally I would be fine with a submodule within abc like abc.interfaces.

Personally I'm starting to warm to the idea of sticking ABCs in proper packages as e.g. functools.abc for Callable; namespaces are one honking great idea after all. :)
msg254861 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2015-11-18 22:46
> I guess is the objection of putting concrete ABCs in the abc module is that the abc module contains the actual code to implement ABCs and not ABCs themselves.

I think it's a rather weak objection.  Having some basic (and fundamental!) ABCs defined along with ABCMeta won't hurt.

> Personally I would be fine with a submodule within abc like abc.interfaces.

> Personally I'm starting to warm to the idea of sticking ABCs in proper packages as e.g. functools.abc for Callable; namespaces are one honking great idea after all. :)

It will be hard to find proper namespace for things like Hashable and Awaitable.  That's why I like the idea of putting them in the top-level 'abc' module.
msg255013 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-20 19:56
I would not like to have, say, 20 namespaces, some with just one ABD, such as functools.abc.Callable.  I might have preferred abc.collections to collections.abc, etc.
msg257204 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-12-29 18:45
I brought this topic up on python-ideas and both Nick Coghlan and MA Lemburg suggested the abc module like Yury did. So the proposal becomes to put syntax-related ABCs into the abc module and domain-specific ones in their respective stdlib modules.

Nick suggested the following ABCs get pulled out of collections.abc and put into the abc module:

    Callable - function calls
    Iterable - for loops, comprehensions
    Iterator - for loops, comprehensions
    Generator - generators, generator expressions
    Awaitable - await expressions
    Coroutine - async def
    AsyncIterable - async for
    AsyncIterator  - async for
msg257214 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-12-29 20:31
That looks like a good list to me.
msg257274 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-01-01 05:12
From a documentation perspective, I'd suggest breaking the ABC module documentation at https://docs.python.org/3/library/abc.html into three subsections:

29.7.1 Defining Abstract Base Classes

This would include the existing docs for ABC, ABCMeta, abstractmethod and get_cache_token.

29.7.2 Syntactic ABCs

New section containing the documentation for the ABCs relocated from collections.abc

29.7.3 Deprecated decorator API

The docs for abstractclassmethod, abstractstaticmethod and abstractproperty can be moved out of the main ABC definition docs, since the only reason to use them now is for single source Python 2/3 compatibility, or compatibility with Python 3.2.
msg257375 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-01-02 22:25
Did anyone consider moving these near the “types” module, either directly inside, or as a “types.abc” submodule? In my mind, these ABCs would fit reasonably well there. They are related to built-in types, but do not have built-in names.
msg257384 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2016-01-02 23:03
> Did anyone consider moving these near the “types” module, either directly inside, or as a “types.abc” submodule? In my mind, these ABCs would fit reasonably well there. They are related to built-in types, but do not have built-in names.

Big -1.  Let's avoid nesting 'abc' modules throughout the standard library.  One of the good practices in Python is to import modules, not classes from modules.  With 'types.abc' I'll have to import 'from types import abc', and then, in the code, I won't be so sure which 'abc' is it -- 'abc', 'types.abc', 'collections.abc' etc.

Putting ABCs in the 'types' module directly also doesn't feel right and can confuse users.  'types' already contains things like 'GeneratorType', 'CoroutineType' etc, which are actual types (not ABCs).
msg257547 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-01-05 20:33
Honestly I think this such a slippery slope that I prefer keeping them in collections.abc. The main reason that we have Iterable and Iterator is that they are closely related to real collections (e.g. Sequence, Set, Mapping).  And generators are related to iterators.  And so on.  To me the force that wants to keep abc.py purely about *implementing* ABCs is stronger than the desire to have only things that really are ABCs for collection types in collections.abc.

The types.py module is a bunch of crap that we can't get rid of yet but it should not be used for anything new.
msg257552 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-05 21:09
So leave Callable and Coroutine there and if I add a context manager ABC put it in collections.abc as well? Or should we put Callable and Coroutine in functools and a context manager one in contextlib (or Coroutine in asyncio)?
msg257553 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-01-05 21:14
I think ContextManager should be in contextlib. I don't want to put
anything in functools. Coroutine doesn't belong in asyncio, it's more
fundamental (closely related to Generator).
msg257554 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-05 21:14
Thanks for the input everyone!
History
Date User Action Args
2016-01-05 21:19:53brett.cannonlinkissue25609 dependencies
2016-01-05 21:14:59brett.cannonsetstatus: open -> closed
resolution: wont fix
messages: + msg257554
2016-01-05 21:14:10gvanrossumsetmessages: + msg257553
2016-01-05 21:09:02brett.cannonsetmessages: + msg257552
2016-01-05 20:33:16gvanrossumsetnosy: + gvanrossum
messages: + msg257547
2016-01-02 23:03:57yselivanovsetmessages: + msg257384
2016-01-02 22:25:26martin.pantersetnosy: + martin.panter
messages: + msg257375
2016-01-01 05:12:35ncoghlansetmessages: + msg257274
2015-12-29 20:31:51terry.reedysetmessages: + msg257214
2015-12-29 18:45:13brett.cannonsetmessages: + msg257204
2015-12-29 03:29:35ncoghlansetnosy: + ncoghlan
2015-11-20 19:56:36terry.reedysetnosy: + terry.reedy
messages: + msg255013
2015-11-18 22:46:01yselivanovsetmessages: + msg254861
2015-11-16 20:12:57brett.cannonsetmessages: + msg254750
2015-11-16 19:59:01yselivanovsetnosy: + yselivanov
messages: + msg254747
2015-11-16 19:23:50ezio.melottisetnosy: + ezio.melotti
2015-11-16 19:21:22brett.cannoncreate