classification
Title: expose _abcoll as collections.abc
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: Laurent.Mazuel, brett.cannon, daniel.urban, eric.araujo, flox, georg.brandl, pitrou, r.david.murray, rhettinger, terry.reedy
Priority: low Keywords: patch

Created on 2011-01-31 19:44 by rhettinger, last changed 2013-02-01 22:14 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
collections.abc-in-docs.diff eric.araujo, 2011-02-26 14:19
Messages (20)
msg127652 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-31 19:44
For the 3.3, make _abcoll (which is full of the collections abstract base classes) visible as a module called collections.abc and document that as the preferred way to access them.

For backwards compatibility, continue to import the names directly into the collections.py namespace so that no one has to change existing code.

Background:  Experience with teaching ABCs and dealing with bug reports has shown that it is creating some confusion having the long list of abstract APIs commingled with the concrete APIs (for example, seeing collections.Mapping and thinking it is one of the various concrete types in the collections module).  If it were to become a practice to write collections.abc.Mapping, it would be immediately clear that an ABC was being used rather than a concrete class like OrderedDict.

The other reason to separate them is that the use cases tend to be different.  People look to the abstract APIs either for a specification (reference purposes), for mixin methods (aid in building their own classes), or for registration (to control isinstance and issubclass).  In contrast, people use concrete APIs directly for managing data.  Accordingly, it makes senses to group the two different types of tools into separate toolboxes.
msg127656 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-01-31 20:18
Why not just put them in the 'abc' namespace?  IMO, collections.abc.Callable makes a lot less sense than abc.Mapping.
msg127662 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-31 20:59
> Why not just put them in the 'abc' namespace?

Two reasons:
* There are lots of ABCs scattered throughout the standard libary that aren't collections ABCs (the importlib ABCs and IO ABCs for example).
* Guido viewed collections ABCs as tightly associated with collections.
msg127667 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-01-31 21:42
Hmm.  OK, so it is just Callable that is the odd man out.

But in that case, shouldn't the pattern be adopted by the other modules that define abcs as well?  And if the distinction isn't sharp enough in those other modules to justify that, then I question whether it is a good idea to special-case collections.  The docs already make the distinction pretty clear.

My "flat is better than nested" alarm is going off here :)
msg127708 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-02-01 20:43
Importlib puts all of its ABCs in importlib.abc, so at least one package has already taken this approach.

I for one support the collections.abc idea.
msg127721 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-02-02 08:15
And what about those "collection" ABCs that aren't collections?  These are at least

* Hashable
* Callable
* ByteString
* Iterator
msg127724 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-02-02 08:23
> And what about those "collection" ABCs that aren't collections?

<shrug> Guido originally thought all these should live together and I don't see much of a win in separating them from the other.
msg127913 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-02-04 17:50
Seems a good idea to me.

Regarding the implementation, knowing your reluctance to turn modules into packages, I guess you’re talking about exposing collections.abc in a similar manner to os.path, which is fine IMO.
msg127963 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-02-05 01:12
However done, I would prefer separation also.
msg127965 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-02-05 01:21
> Regarding the implementation, knowing your reluctance to turn modules
> into packages, I guess you’re talking about exposing collections.abc in 
> a similar manner to os.path, which is fine IMO.

That makes things confusing:

>>> import os.path
>>> os.path.__name__
'posixpath'

>>> import collections
>>> collections.Mapping
<class '_abcoll.Mapping'>

yikes?
msg127969 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-02-05 02:11
Okay, I plead guilty of premature implementation talk.  The clean solution is just moving collections.py to collections/__init__.py and _abcoll.py to collections/abc.py, but I will defer to Raymond here.
msg128027 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-02-05 21:31
I'll use the packaging approach.  The os.path technique predated packages and is no longer the preferred way of doing things.
msg129016 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-02-22 00:44
Followed Brett's example with importlib and made collection into a package with an abc module.  See r88490.
msg129552 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-02-26 14:19
Some missed doc changes.
msg145073 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-10-07 12:58
For the record, this made unloaded interpreter startup quite a bit slower since importing one of the abcs now imports the whole collection package and its dependencies.
From 3.2 to 3.3:

### normal_startup ###
Min: 0.842961 -> 1.091329: 1.29x slower
Avg: 0.851607 -> 1.106344: 1.30x slower
Significant (t=-37.61)
Stddev: 0.00623 -> 0.01381: 2.2162x larger
Timeline: http://tinyurl.com/3etyx44

### startup_nosite ###
Min: 0.247490 -> 0.378279: 1.53x slower
Avg: 0.255694 -> 0.382722: 1.50x slower
Significant (t=-39.46)
Stddev: 0.00865 -> 0.00536: 1.6141x smaller
Timeline: http://tinyurl.com/3vovjwb

It probably doesn't make much difference in practice, since collections is one of those modules everyone will use in their code.
msg145138 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-10-07 20:05
> collections is one of those modules everyone will use in their code.

Simply untrue, and definitely not in every program. It is also irrelevant for bringing up the interactive interpreter, where there initially is no user code and which should happen as fast as possible. I also doubt IDLE uses collections to bring up its shell window.
msg145139 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-10-07 20:11
> > collections is one of those modules everyone will use in their code.
> 
> Simply untrue, and definitely not in every program. It is also
> irrelevant for bringing up the interactive interpreter, where there
> initially is no user code and which should happen as fast as possible.

I don't think a 50ms startup time is a problem for the interactive
interpreter. Of course, it will be more noticeable on very slow
machines.

> I also doubt IDLE uses collections to bring up its shell window.

The collections module appeared in 2.4; antiquated code might indeed not
know about it ;)
msg145142 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-10-07 20:36
You also have to think about command-line apps like Mercurial that need to be snappy.  For those guys, a lot of added startup time is a big deal -- one of the reasons Matt Mackall hates thinking about a Python 3 port is that Python 3(.2) startup time is already double that of Python 2.
msg145150 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-10-07 22:52
A couple of years ago, various people worked hard to remove unneeded imports from the interpreter startup routine. The result was both quite noticeable and much appreciated on my old xp machine. I hate to see much of that progress tossed away.
msg181123 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-02-01 22:14
I'm closing this again as 3.3 actually starts up faster than 3.2 thanks to importlib so this slowdown is no longer noticeable.
History
Date User Action Args
2013-02-01 22:14:05brett.cannonsetstatus: open -> closed

messages: + msg181123
2012-03-15 13:45:33Laurent.Mazuelsetnosy: + Laurent.Mazuel
2011-10-18 13:12:08floxsetnosy: + flox
2011-10-07 23:48:57rhettingersetassignee: rhettinger
2011-10-07 22:52:57terry.reedysetmessages: + msg145150
2011-10-07 20:36:22georg.brandlsetstatus: closed -> open

messages: + msg145142
2011-10-07 20:11:58pitrousetmessages: + msg145139
2011-10-07 20:05:18terry.reedysetmessages: + msg145138
2011-10-07 12:58:07pitrousetmessages: + msg145073
2011-02-26 14:19:16eric.araujosetfiles: + collections.abc-in-docs.diff

messages: + msg129552
keywords: + patch
nosy: brett.cannon, georg.brandl, rhettinger, terry.reedy, pitrou, eric.araujo, r.david.murray, daniel.urban
2011-02-22 00:44:14rhettingersetstatus: open -> closed

messages: + msg129016
resolution: fixed
nosy: brett.cannon, georg.brandl, rhettinger, terry.reedy, pitrou, eric.araujo, r.david.murray, daniel.urban
2011-02-05 21:31:41rhettingersetnosy: brett.cannon, georg.brandl, rhettinger, terry.reedy, pitrou, eric.araujo, r.david.murray, daniel.urban
messages: + msg128027
2011-02-05 02:11:39eric.araujosetnosy: brett.cannon, georg.brandl, rhettinger, terry.reedy, pitrou, eric.araujo, r.david.murray, daniel.urban
messages: + msg127969
2011-02-05 01:21:28pitrousetnosy: + pitrou
messages: + msg127965
2011-02-05 01:12:56terry.reedysetnosy: + terry.reedy
messages: + msg127963
2011-02-04 17:50:57eric.araujosetnosy: + eric.araujo
messages: + msg127913
2011-02-02 08:23:32rhettingersetnosy: brett.cannon, georg.brandl, rhettinger, r.david.murray, daniel.urban
messages: + msg127724
2011-02-02 08:15:52georg.brandlsetnosy: + georg.brandl
messages: + msg127721
2011-02-01 20:43:05brett.cannonsetnosy: + brett.cannon
messages: + msg127708
2011-02-01 09:55:40daniel.urbansetnosy: + daniel.urban
2011-01-31 21:42:15r.david.murraysetmessages: + msg127667
2011-01-31 20:59:08rhettingersetmessages: + msg127662
2011-01-31 20:18:21r.david.murraysetnosy: + r.david.murray
messages: + msg127656
2011-01-31 19:44:25rhettingercreate