classification
Title: Save OrderedDict imports in various stdlibs.
Type: performance Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: 32336 32338 Superseder:
Assigned To: Nosy List: methane, miss-islington, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2017-12-18 09:45 by methane, last changed 2018-04-03 06:12 by methane. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4994 merged thatiparthy, 2017-12-23 20:04
PR 5000 closed thatiparthy, 2017-12-23 22:00
PR 5001 merged methane, 2017-12-24 05:54
PR 5013 closed thatiparthy, 2017-12-26 04:15
PR 5315 merged methane, 2018-01-25 10:32
PR 6361 merged miss-islington, 2018-04-03 03:41
Messages (9)
msg308538 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-12-18 10:00
csv and re has issues already.
There are some remaining OrderdDict, including docstring.
It looks like some of them can be removed easily without doc update.

This is `ag --exclude-dir=test OrderedDict`, exluding csv, re, pprint and collections.

typing:
2109:    nm_tpl.__annotations__ = nm_tpl._field_types = collections.OrderedDict(types)

unittest/util.py
3:from collections import namedtuple, OrderedDict
158:    c = OrderedDict()

enum.py
153:        enum_class._member_map_ = OrderedDict()      # name->value map
615:        # We use an OrderedDict of sorted source keys so that the

lib2to3/pgen2/grammar.py
91:        dump() recursively changes all dict to OrderedDict, so the pickled file
93:        pickled file to create the tables, but  only changes OrderedDict to dict
94:        at the top level; it does not recursively change OrderedDict to dict.
96:        passed to load() in that some of the OrderedDict (from the pickled file)
98:        performance because OrderedDict uses dict's __getitem__ with nothing in
142:        return collections.OrderedDict(

asyncio/base_events.py
842:                addr_infos = collections.OrderedDict()

configparser.py
142:from collections import OrderedDict as _default_dict, ChainMap as _ChainMap

inspect.py
52:from collections import namedtuple, OrderedDict
1712:    new_params = OrderedDict(old_params.items())
2563:    * arguments : OrderedDict
2663:        self.arguments = OrderedDict(new_arguments)
2694:    * parameters : OrderedDict
2724:            params = OrderedDict()
2727:                params = OrderedDict()
2762:                params = OrderedDict(((param.name, param)
2839:        arguments = OrderedDict()

json/tool.py
41:                                object_pairs_hook=collections.OrderedDict)

json/__init__.py
31:    >>> from collections import OrderedDict
32:    >>> mydict = OrderedDict([('4', 5), ('6', 7)])
290:    collections.OrderedDict will remember the order of insertion). If
318:    collections.OrderedDict will remember the order of insertion). If

json/decoder.py
297:        collections.OrderedDict will remember the order of insertion). If

email/_header_value_parser.py
73:from collections import OrderedDict
720:        params = OrderedDict()

pydoc_data/topics.py
8585:                 '"collections.OrderedDict" to remember the order that class '
8593:                 '           return collections.OrderedDict()\n'
8614:                 'empty "collections.OrderedDict".  That mapping records the '
msg308974 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-12-23 23:09
These should be looked at one at a time (no sweeping search and replace missions).

The original author should be consulted when possible (they are in a better position to judge whether the original intent was preserved).

Published APIs should not be changed (i.e. you can't just change the default argument for configparser).

Docs should be changed carefully (i.e. let Bob Ippolito decide whether he wants to change the examples of how to use the object_pairs_hook).

Guido was very clear that you can't just downgrade every occurrence of OrderedDict with dict.  Please don't be so aggressive (grepping every occurrence in the standard library without giving it individual consideration).
msg308983 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-12-24 05:48
Agreed.
I won't remove OrderedDict usages which is part of public APIs.
This issue is for finding and removing easy local usages.
msg308984 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2017-12-24 06:07
Let's remove from these:

* asyncio/base_events.py:  Used for local variable. 
* email/_header_value_parser.py: Used for local variable.
* json: Examples (in document and docstring) for keeping order, and json.tool.

I don't touch other modules.
msg308987 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-24 08:18
I concur with Raymond. I made the same searching before and have found that there are not much opportunities for getting rid of OrderedDict and there are even less cases that will get a benefit from this.
msg309008 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-12-24 18:34
> * asyncio/base_events.py:  Used for local variable. 
> * email/_header_value_parser.py: Used for local variable.
> * json: Examples (in document and docstring) for keeping order, and json.tool.

All of these modules have active maintainers.  You should create separate issues for each one of these and assign to those maintainers.  You don't just get to mow down all the other developers.  That isn't how we work together.

In the JSON module, there does need to be an example of how to use object_pairs_hook, so you will need to come up with a replacement or an additional note explaining in Py3.7 and later, most of the OrderedDict use cases have been supplanted by the regular dict.
msg310670 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-01-25 10:53
New changeset 2812d3d99287c50bab99625d7240bcf1c2e32369 by INADA Naoki in branch 'master':
bpo-32360: Remove OrderedDict usage from json.tool (GH-5315)
https://github.com/python/cpython/commit/2812d3d99287c50bab99625d7240bcf1c2e32369
msg314858 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-04-03 03:39
New changeset 629338f1404ea9cd75e4fb0389a2fbe1b589de43 by INADA Naoki in branch 'master':
bpo-32360: Remove object_pairs_hook=OrderedDict examples (GH-5001)
https://github.com/python/cpython/commit/629338f1404ea9cd75e4fb0389a2fbe1b589de43
msg314860 - (view) Author: miss-islington (miss-islington) Date: 2018-04-03 04:31
New changeset 0c533573c527c63dacab5921becb66dc3eb274c1 by Miss Islington (bot) in branch '3.7':
bpo-32360: Remove object_pairs_hook=OrderedDict examples (GH-5001)
https://github.com/python/cpython/commit/0c533573c527c63dacab5921becb66dc3eb274c1
History
Date User Action Args
2018-04-03 06:12:08methanesetstatus: open -> closed
dependencies: - Dict order is now guaranteed, so add tests and doc for it
stage: patch review -> resolved
2018-04-03 05:29:09methanesetresolution: fixed
2018-04-03 04:31:30miss-islingtonsetnosy: + miss-islington
messages: + msg314860
2018-04-03 03:41:56miss-islingtonsetpull_requests: + pull_request6072
2018-04-03 03:39:49methanesetmessages: + msg314858
2018-01-25 10:53:02methanesetmessages: + msg310670
2018-01-25 10:32:09methanesetpull_requests: + pull_request5161
2017-12-26 04:15:51thatiparthysetpull_requests: + pull_request4902
2017-12-24 18:34:21rhettingersetmessages: + msg309008
2017-12-24 08:18:24serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg308987
2017-12-24 06:07:59methanesetmessages: + msg308984
2017-12-24 05:54:53methanesetpull_requests: + pull_request4890
2017-12-24 05:48:13methanesetmessages: + msg308983
2017-12-23 23:09:33rhettingersetnosy: + rhettinger
messages: + msg308974
2017-12-23 22:00:05thatiparthysetpull_requests: + pull_request4889
2017-12-23 20:04:40thatiparthysetkeywords: + patch
stage: patch review
pull_requests: + pull_request4883
2017-12-18 10:00:10methanesetmessages: + msg308538
2017-12-18 09:53:57serhiy.storchakasetdependencies: + Dict order is now guaranteed, so add tests and doc for it
2017-12-18 09:52:27serhiy.storchakasetdependencies: + Save OrderedDict import in argparse, Save OrderedDict import in re
type: performance
components: + Library (Lib)
versions: + Python 3.7
2017-12-18 09:45:46methanecreate