classification
Title: pprint for dict in sorted order or insert order?
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Aaron Hall, Jan-Hein Bührman, Valentin Beyer, eamanu, ewjoachim, josephsmeng, pombreda, remi.lapeyre, rhettinger, serhiy.storchaka, xtreak
Priority: normal Keywords: patch, patch, patch

Created on 2017-06-15 02:15 by josephsmeng, last changed 2019-03-23 01:17 by xavierguihot. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11769 merged remi.lapeyre, 2019-02-06 11:08
PR 11769 merged remi.lapeyre, 2019-02-06 11:08
PR 11769 merged remi.lapeyre, 2019-02-06 11:08
PR 12500 merged xavierguihot, 2019-03-23 01:17
Messages (19)
msg296056 - (view) Author: Joseph Shen (josephsmeng) * Date: 2017-06-15 02:15
start from 3.6+, dict keys are ordered heir creation order.
the builtin print function works as we expected, but for 
pprint, the keys is sorted. should we using the sorted version
or just obey the creation order as builtin print?
msg296065 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-06-15 04:25
Until dict insertion order is guaranteed, it doesn't make sense to change pprint().
msg296068 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-15 05:24
Concur with Raymond.
msg305440 - (view) Author: Aaron Hall (Aaron Hall) * Date: 2017-11-02 16:34
If/when order is guaranteed (3.7?) we should have a pprint that respects current order, 

-or-

we should get an improved pprint (maybe named pp or print?) that understands mappings and other abstract data types.

I had a conversation about pprint at the Python meetup last night. It kinda went like this: https://www.youtube.com/watch?v=NpYEJx7PkWE 

Maybe now's the time for improved behavior?
msg305442 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-11-02 17:02
A year ago I proposed to add a method for detecting whether a dict preserves an order and in what sense. [1] But this idea was rejected.

[1] https://mail.python.org/pipermail/python-dev/2016-October/146597.html
msg310898 - (view) Author: Philippe (pombreda) Date: 2018-01-27 23:14
IMHO since Guido said that dictionary order is guaranteed going forward [1], the order should now be preserved in 3.6+ 

Getting a sorted pprint'ed dict in a Py3.6.1 was a surprise to me coming from Python2.

The default ordered dict is to me the killer feature of 3 and a major driver to migrate from 2.

[1] https://mail.python.org/pipermail/python-dev/2017-December/151286.html
msg327904 - (view) Author: Jan-Hein Bührman (Jan-Hein Bührman) Date: 2018-10-17 15:17
@terry.reedy - If I understand correctly, this issue was closed by you awaiting what would happen with the dict insertion order after 3.6.

It has been decided that dict insertion will stay for now and in the future. >>> Should this issue now be reopened?

And I would favor backward compatibility (key sort order) with an additional keyword for pprint to display it "unsorted" order (from pprint perspective, insertion order for dicts and OrderedDicts, and whatever other order for whatever other dicts).
msg334588 - (view) Author: Valentin Beyer (Valentin Beyer) Date: 2019-01-30 19:08
Please reopen this.
With version 3.7 and guaranteed dict order pprint should not sort by key.
msg334605 - (view) Author: Joseph Shen (josephsmeng) * Date: 2019-01-31 05:57
I reopened this issue, and change affected versions to 3.6, 3.7 and 3.8.
msg334689 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2019-02-01 15:34
Hi josephsmeng, dict order preservation is now used in a lot of place in Python. I will post a patch to make pprint use insertion order in a few hours.
msg334693 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-02-01 16:02
https://docs.python.org/3/library/pprint.html

> Dictionaries are sorted by key before the display is computed.

https://mail.python.org/pipermail/python-dev/2017-December/151369.html . There was some discussion in the thread about this being a breaking change and the behavior of pprint. @remi.lapeyre I would suggest waiting for a core dev on approving this change before proceeding with a patch to avoid work being wasted.
msg334846 - (view) Author: Joachim Jablon (ewjoachim) Date: 2019-02-04 21:57
If you stop sorting keys in pprint, then unittest.TestCase.assertDictEquals() (and pytest and such) won't be able to generate a meaningful diff when 2 dicts aren't equal, and it will be much more complicated to understand why a test fails.

https://github.com/python/cpython/blob/89427cd0feae25bbc8693abdccfa6a8c81a2689c/Lib/unittest/case.py#L1172-L1175

(well except if we fix unittest, but then external test runners will need fixing too, and it might not be the only place diff is used on the result of pprint to see how two objects differ)
msg334854 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-02-05 01:52
Consider adding a flag to switch between sorted order and insertion order (defaulting to the current behavior).  For convenience, add a short-cut function to call pprint() with the flag set to False.

    def pprint(data, *, sort_dicts=True):
        ...

    def pp(data, *args, **kwds):
        pprint(data, *args, sort_dicts=False, **kwds)
msg334932 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-02-06 13:09
Hi!

>    def pprint(data, *, sort_dicts=True):
>        ...

>    def pp(data, *args, **kwds):
>        pprint(data, *args, sort_dicts=False, **kwds)

We could use the parameters `sort=`? IMHO is better a short parameters name.

If we assign by default sort_dicts=True, that would have some error on unittest, isn't?
msg334933 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-02-06 13:14
pprint only sorts dicts currently so sort_dicts is better than sort that sounds more general. sort_dicts=True is the current behavior so I guess this is backwards compatible way to do this change.
msg334936 - (view) Author: Emmanuel Arias (eamanu) * Date: 2019-02-06 13:20
> pprint only sorts dicts currently so sort_dicts is better than sort that sounds more general. 

Yes. Because pprint just sort dicts, I think that is better rename the parameter like just sort. But was an opinion, that does not affect to the behavior. 

> sort_dicts=True is the current behavior so I guess this is backwards compatible way to do this change.

I understand it. I was confuse.
msg334937 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2019-02-06 13:20
I agree with Karthikeyan, I would expect sort=True to sort all collections, including tuples and lists for example.
msg338614 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-03-22 17:22
New changeset 96831c7fcf888af187bbae8254608cccb4d6a03c by Raymond Hettinger (Rémi Lapeyre) in branch 'master':
bpo-30670: Add pp function to the pprint module (GH-11769)
https://github.com/python/cpython/commit/96831c7fcf888af187bbae8254608cccb4d6a03c
msg338615 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-03-22 17:24
Thanks for the contribution.
History
Date User Action Args
2019-03-23 01:17:06xavierguihotsetpull_requests: + pull_request12455
2019-03-22 17:24:37rhettingersetstatus: open -> closed
messages: + msg338615

keywords: patch, patch, patch
resolution: fixed
stage: patch review -> resolved
2019-03-22 17:22:23rhettingersetmessages: + msg338614
2019-02-06 13:20:44remi.lapeyresetmessages: + msg334937
2019-02-06 13:20:25eamanusetmessages: + msg334936
2019-02-06 13:14:50xtreaksetkeywords: patch, patch, patch

messages: + msg334933
2019-02-06 13:09:05eamanusetnosy: + eamanu
messages: + msg334932
2019-02-06 11:09:11remi.lapeyresetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request11734
2019-02-06 11:08:55remi.lapeyresetkeywords: + patch
stage: needs patch -> needs patch
pull_requests: + pull_request11733
2019-02-06 11:08:36remi.lapeyresetkeywords: + patch
stage: needs patch -> needs patch
pull_requests: + pull_request11732
2019-02-05 03:22:15xtreaksetstage: resolved -> needs patch
2019-02-05 01:52:27rhettingersetresolution: later -> (no value)
messages: + msg334854
versions: - Python 3.6, Python 3.7
2019-02-04 21:57:48ewjoachimsetnosy: + ewjoachim
messages: + msg334846
2019-02-01 16:02:26xtreaksetnosy: + xtreak
messages: + msg334693
2019-02-01 15:34:17remi.lapeyresetnosy: + remi.lapeyre
messages: + msg334689
2019-01-31 05:57:00josephsmengsetstatus: closed -> open

messages: + msg334605
versions: + Python 3.8
2019-01-30 19:08:25Valentin Beyersetnosy: + Valentin Beyer
messages: + msg334588
2018-10-17 15:17:43Jan-Hein Bührmansetnosy: + Jan-Hein Bührman
messages: + msg327904
2018-01-27 23:14:39pombredasetnosy: + pombreda
messages: + msg310898
2017-11-02 17:02:35serhiy.storchakasetmessages: + msg305442
2017-11-02 16:34:21Aaron Hallsetnosy: + Aaron Hall
messages: + msg305440
2017-06-17 01:51:44terry.reedysetstatus: open -> closed
resolution: later
stage: resolved
2017-06-15 05:24:05serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg296068
2017-06-15 04:25:33rhettingersetnosy: + rhettinger
messages: + msg296065
2017-06-15 02:15:55josephsmengcreate