msg295001 - (view) |
Author: Eric O. LEBIGOT (lebigot) |
Date: 2017-06-02 09:47 |
The JSON encoder for dictionaries preserves the order of the items in a dictionary: it would be useful to document this behavior, so that users can rely on it.
While JSON itself does not have ordered key/value pairs, this feature can be useful (for modifying the original JSON document while maintaining as much of it untouched, for manipulating our own extension of JSON with ordered dictionaries, etc.).
The documentation could also usefully mention that _reading_ sets of key/value pairs can also be done in an order-preserving way with `object_pairs_hook=OrderedDict`.
Reference: the json module does not alter the order of the dictionary items upon encoding: https://github.com/python/cpython/blob/aacd53f6cb96fe8c4fe9ce894f22e25f356a97c3/Lib/json/encoder.py#L355.
|
msg347712 - (view) |
Author: Josh Rosenberg (josh.r) * ![Python triager (Python triager)](@@file/triager.png) |
Date: 2019-07-11 22:36 |
Is there a reason to document object_pairs_hook=OrderedDict rather than just making the decoder populate a regular dict in an order-preserving way? (No idea if it does this already)
|
msg347857 - (view) |
Author: Kyle Stanley (aeros) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-13 21:57 |
> The JSON encoder for dictionaries preserves the order of the items in a dictionary
From what I can tell, the JSON encoder does nothing to preserve the order of the keys. Although the default option is to not modify the existing dictionary, the items() method returns the k,v pairs in a non-random arbitrary manner, which may not match the order the values were entered in. See https://docs.python.org/2/library/stdtypes.html#dict.items.
|
msg347938 - (view) |
Author: Eric O. LEBIGOT (lebigot) |
Date: 2019-07-14 20:06 |
Kyle, what you're saying is correct but is unfortunately unrelated to any of the points in the original issue. In fact, the JSON encoder does preserve whatever order the dictionary elements are in, and it would be useful to document this. Thus, if a user gives an OrderedDict (or a dict with a known order, in Python 3.7+), it is useful that he know that the order of its elements will not be changed upon transformation into JSON.
|
msg348227 - (view) |
Author: Kyle Stanley (aeros) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-21 07:07 |
>Thus, if a user gives an OrderedDict (or a dict with a known order, in Python 3.7+), it is useful that >he know that the order of its elements will not be changed upon transformation into JSON.
I would agree that it could be helpful to document that if a user passes an OrderedDict, the order of the elements will not changed, I was just pointing out that a default dictionary implementation would not normally retain the order.
However, after looking through the 3.7 changes and reading the discussions, it looks like dictionaries retaining insertion order was decided to be made an official part of the language. If the order matters though, it would make sense to recommend users to use OrderedDict. The order in the default dictionary implementation can end up becoming sorted if pprint is used.
Discussions: https://mail.python.org/pipermail/python-dev/2017-December/151283.html and https://mail.python.org/pipermail/python-dev/2017-December/151376.html
|
msg348232 - (view) |
Author: Inada Naoki (methane) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-21 08:52 |
OrderedDict is not recommended to preserve order any more.
Note that mention about `object_pairs_hook=OrderedDict` is removed intentionally.
https://github.com/python/cpython/pull/5001
Please forget about OrderedDict unless you need additional feature OrderedDict provides. It is far inefficient than regular dict.
|
msg348239 - (view) |
Author: Eric O. LEBIGOT (lebigot) |
Date: 2019-07-21 12:18 |
The essence of the original post is simply to document any order-preserving properties of the Python to JSON and JSON to Python transformation for mappings. This way users can rely on it. This is for instance useful if a program modifies JSON created by a user and wants to preserve the user ordering.
|
msg348265 - (view) |
Author: Kyle Stanley (aeros) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-21 20:51 |
> OrderedDict is not recommended to preserve order any more.
> Please forget about OrderedDict unless you need additional feature OrderedDict provides. It is far inefficient than regular dict.
Thanks for the clarification. Should this recommendation be briefly mentioned in the documentation (https://docs.python.org/3.9/library/collections.html#ordereddict-objects)?
Currently, the docs reference that as of 3.7, the order-preserving behavior of standard dictionaries is guaranteed, but there is no mention of the significant differences in efficiency. Based on the current description, a user might end up using OrderedDictionary on the basis that they *might* have a need for reordering elements.
If the difference in performance is significant enough, it would be worth noting in the docs so that users can take it into consideration when deciding which data structure to use.
|
msg348275 - (view) |
Author: Raymond Hettinger (rhettinger) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-22 01:09 |
Kyle, the link you gave already discusses performance to the extent that it matters. Please keep focused on the OP's thought that the JSON docs should explicitly promised that key/value pairs are added in the order that they are encountered. The existing JSON docs imply this pretty strongly, but it can be made more explicit.
|
msg348276 - (view) |
Author: Kyle Stanley (aeros) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-07-22 02:06 |
> Please keep focused on the OP's thought that the JSON docs should explicitly promised that
> key/value pairs are added in the order that they are encountered. The existing JSON docs imply
> this pretty strongly, but it can be made more explicit.
I agree that the behavior could be more explicitly stated in the docs, as that could be potentially useful information for users to be aware of.
Apologies for derailing the topic, I became too focused on the OrderedDict vs Dict topic, which was only a small part of the issue. Any further discussion on that topic or updating the OrderedDict docs should probably be moved into it's own issue, if at all.
|
msg350234 - (view) |
Author: Raymond Hettinger (rhettinger) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-08-22 22:14 |
New changeset 657008ea0336ff4f275ed3f0c2b6dd2e52de2bba by Raymond Hettinger in branch 'master':
bpo-30550: Clarify JSON ordering guarantees (GH-15397)
https://github.com/python/cpython/commit/657008ea0336ff4f275ed3f0c2b6dd2e52de2bba
|
msg350237 - (view) |
Author: Raymond Hettinger (rhettinger) * ![Python committer (Python committer)](@@file/committer.png) |
Date: 2019-08-22 22:50 |
New changeset 4a40498ea96a3c606952712c7951b2ea4ab258e4 by Raymond Hettinger (Miss Islington (bot)) in branch '3.8':
bpo-30550: Clarify JSON ordering guarantees (GH-15397) (GH-15403)
https://github.com/python/cpython/commit/4a40498ea96a3c606952712c7951b2ea4ab258e4
|
|
Date |
User |
Action |
Args |
2022-04-11 14:58:47 | admin | set | github: 74735 |
2019-08-22 22:51:19 | rhettinger | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
2019-08-22 22:50:37 | rhettinger | set | messages:
+ msg350237 |
2019-08-22 22:15:49 | miss-islington | set | pull_requests:
+ pull_request15110 |
2019-08-22 22:14:47 | rhettinger | set | messages:
+ msg350234 |
2019-08-22 17:29:28 | rhettinger | set | keywords:
+ patch stage: patch review pull_requests:
+ pull_request15105 |
2019-07-22 02:06:39 | aeros | set | messages:
+ msg348276 |
2019-07-22 01:10:11 | rhettinger | set | priority: normal -> low versions:
- Python 3.7 |
2019-07-22 01:09:34 | rhettinger | set | messages:
+ msg348275 |
2019-07-22 00:26:44 | rhettinger | set | assignee: docs@python -> rhettinger |
2019-07-21 20:51:38 | aeros | set | messages:
+ msg348265 |
2019-07-21 12:18:55 | lebigot | set | messages:
+ msg348239 |
2019-07-21 08:52:49 | methane | set | nosy:
+ methane messages:
+ msg348232
|
2019-07-21 07:07:41 | aeros | set | messages:
+ msg348227 |
2019-07-14 20:06:39 | lebigot | set | messages:
+ msg347938 |
2019-07-13 21:57:40 | aeros | set | nosy:
+ aeros messages:
+ msg347857
|
2019-07-11 22:36:47 | josh.r | set | nosy:
+ josh.r messages:
+ msg347712
|
2019-07-11 06:37:15 | xtreak | set | nosy:
+ rhettinger
versions:
+ Python 3.7, Python 3.8, Python 3.9, - Python 3.6 |
2019-07-11 06:16:47 | Dima.Tisnek | set | nosy:
+ Dima.Tisnek
|
2017-06-02 09:49:46 | lebigot | set | assignee: docs@python
components:
+ Documentation, - Library (Lib) nosy:
+ docs@python |
2017-06-02 09:48:20 | lebigot | set | title: Document order-preserving dictionary output in son -> Document order-preserving dictionary output in json |
2017-06-02 09:48:14 | lebigot | set | title: Document order-preserving dictionary output -> Document order-preserving dictionary output in son |
2017-06-02 09:47:27 | lebigot | create | |