classification
Title: add json.tool option to avoid alphabetic sort of fields
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: berker.peksag Nosy List: Pavel.Kazlou, berker.peksag, ezio.melotti, martin.panter, pitrou, python-dev, r.david.murray, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2014-06-03 10:11 by Pavel.Kazlou, last changed 2014-11-10 07:58 by berker.peksag. This issue is now closed.

Files
File name Uploaded Description Edit
issue21650.diff berker.peksag, 2014-06-03 10:44 review
issue21650_v2.diff berker.peksag, 2014-06-14 14:39 review
issue21650_v3.diff berker.peksag, 2014-11-02 17:59 review
issue21650_v4.diff berker.peksag, 2014-11-03 10:44 review
Messages (18)
msg219675 - (view) Author: Pavel Kazlou (Pavel.Kazlou) Date: 2014-06-03 10:11
Currently when you use json.tool, fields are reordered alphabetically.
In source code the value of sort_keys is hardcoded to be true.
It should be easy to expose this option as command line parameter.
msg219677 - (view) Author: Pavel Kazlou (Pavel.Kazlou) Date: 2014-06-03 10:12
This is the line in module I'm talking about:
json.dump(obj, outfile, sort_keys=True, indent=4)
msg219678 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-06-03 10:44
Here's a patch with a test case.
msg219804 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-06-05 11:59
I don't really understand the point of this. The "unsorted" output order will be unpredictable for the user (it isn't necessarily the same as the order of fields in the input data).
msg219834 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-05 19:05
It should be possible to also change the tool to use OrderDicts, though.
msg219835 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-05 19:05
Or does the data get decoded to a dict *before* it gets passed to the object_hook?  Probably, in which case nevermind...
msg219889 - (view) Author: Pavel Kazlou (Pavel.Kazlou) Date: 2014-06-06 16:27
The idea is to keep the same order as in input.
msg219891 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-06 16:44
Yes but the input is turned into a dict, and dicts do not preserve order.  Further, what is passed to the object_hook is already a dict, so the order is already lost before object_hook is called.

Since the parser (or at least the Python version of the parser) first turns the input into a list of pairs, and a Python OrderedDict can be instantiated from a list of pairs, it would theoretically be possible to extend the object_hook API to allow an OrderedDict to be used on decode.  Exactly how to do that so as to retain backward compatibility is a bit of a question.

So, it is *possible* to achieve your aim, but it isn't as simple as allowing sort= to be set False.

IMO being able to preserve the order of the input when desired (ie: use an OrderedDict object_hook) would be a nice feature to have.
msg219893 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-06 16:46
Wait, I read the code wrong.

You can define object_pairs_hook, and use that to return an OrderedDict.  So it should be possible to do this without changing the json module itself.  This is actually documented as a place to use OrderedDict.  Guess I should have read the docs instead of the code :)
msg220556 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-06-14 14:39
Updated patch attached based on feedback from David. Thanks!
msg230431 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-11-01 07:20
To me, "--unsorted" implies arbitrary ordering such as the order generated by regular dictionaries.  I think the option should be "--order-preserving" or some such.  The option name needs to make it clear that the output is in the same order an the input.

As side from the option name, I'm +1 on the patch.  It is very annoying to look at scrambled JSON when the original presentation order made more logical sense.
msg230433 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-01 08:12
May be make sorting keys not default?
msg230434 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-11-01 08:16
> Maybe make sorting keys not default?

If you mean preserve order by default, then yes that would be a nice default.
msg230511 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-11-02 17:59
Thanks for the suggestions.

> If you mean preserve order by default, then yes that would be a nice default.

issue21650_v3.diff implements this idea.
msg230563 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-11-03 20:49
LGTM.
msg230669 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-11-05 08:45
The patch looks good.  One nit, the phrase "sorted by their key" has an odd ring to it and is mildly confusing, though technically correct.  Perhaps, "sorted alphabetically by key" would be better for most folks.
msg230939 - (view) Author: Roundup Robot (python-dev) Date: 2014-11-10 07:56
New changeset 58a871227e5b by Berker Peksag in branch 'default':
Issue #21650: Add an `--sort-keys` option to json.tool CLI.
https://hg.python.org/cpython/rev/58a871227e5b
msg230940 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-11-10 07:58
Thanks for the reviews.
History
Date User Action Args
2014-11-10 07:58:05berker.peksagsetstatus: open -> closed
resolution: fixed
messages: + msg230940

stage: commit review -> resolved
2014-11-10 07:56:43python-devsetnosy: + python-dev
messages: + msg230939
2014-11-05 20:13:04serhiy.storchakasetassignee: berker.peksag
2014-11-05 08:45:58rhettingersetmessages: + msg230669
2014-11-03 20:49:41serhiy.storchakasetmessages: + msg230563
stage: patch review -> commit review
2014-11-03 10:44:06berker.peksagsetfiles: + issue21650_v4.diff
2014-11-02 17:59:52berker.peksagsetfiles: + issue21650_v3.diff

messages: + msg230511
2014-11-01 08:16:33rhettingersetmessages: + msg230434
2014-11-01 08:12:02serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg230433
2014-11-01 07:20:23rhettingersetmessages: + msg230431
2014-10-27 02:48:19martin.pantersetnosy: + martin.panter
2014-06-14 14:39:04berker.peksagsetfiles: + issue21650_v2.diff

messages: + msg220556
stage: needs patch -> patch review
2014-06-06 16:46:49r.david.murraysetmessages: + msg219893
2014-06-06 16:44:19r.david.murraysetmessages: + msg219891
stage: patch review -> needs patch
2014-06-06 16:27:16Pavel.Kazlousetmessages: + msg219889
2014-06-05 19:05:50r.david.murraysetmessages: + msg219835
2014-06-05 19:05:01r.david.murraysetnosy: + r.david.murray
messages: + msg219834
2014-06-05 11:59:58pitrousetnosy: + rhettinger, pitrou, ezio.melotti
messages: + msg219804
2014-06-03 10:45:01berker.peksagsetfiles: + issue21650.diff

versions: + Python 3.5, - Python 2.7
keywords: + patch
nosy: + berker.peksag

messages: + msg219678
stage: patch review
2014-06-03 10:12:52Pavel.Kazlousetmessages: + msg219677
2014-06-03 10:11:31Pavel.Kazloucreate