classification
Title: Fix evaluation order of keys/values in dict comprehensions
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: EvKounis, Jim Fasarakis-Hilliard, janzert, ncoghlan, rhettinger, xiang.zhang
Priority: normal Keywords:

Created on 2017-02-25 19:54 by Jim Fasarakis-Hilliard, last changed 2018-04-29 01:15 by janzert.

Messages (4)
msg288578 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2017-02-25 19:54
Reported from [1] and similar to issue11205 

Currently the evaluation order for keys and values in a dictionary comprehension follows that of assignments. The values get evaluated first and then the keys:

    def printer(v):
        print(v, end=' ')
        return v

    d = {printer(i): printer(j) for i, j in [(1, 2), (3, 4)]}
    # 2 1 4 3

This seems to conflict with the semantics as described in the Semantics section of PEP 274 [2] and according to my interpretation of the reference manual (I'd expect the evaluation to be similar to dict-displays).

How should this be addressed? Fix the evaluation order or specify this edge case an "Implementation detail" in the reference manual?

I already have a fix for this lying around (changes to `compiler_sync_comprehension_generator`, `compiler_sync_comprehension_generator` and a switch in `MAP_ADD`) and can make a pull request if required.

I'm not sure if this is classified as a bug per-se so I only tagged Py3.7 for it.

 [1] http://stackoverflow.com/questions/42201932/order-of-operations-in-a-dictionary-comprehension
 [2] https://www.python.org/dev/peps/pep-0274/#semantics
msg288623 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-02-27 09:35
I think the current behavior is correct and desirable (as you say, it follows the order that would take place in an assignment, making it easy to roll-up existing for-loop code into a dict comprehension or to unroll and existing comprehension). Also, I think changing the behavior might risk introducing bugs into existing code that may have unconsciously relied on the existing order.  My recommendation is to document the current value-first behavior.

For the other issue, 11205, I agree with the discussion there that key-first-value-second makes more sense in the context of literals which are normally evaluated left-to-right.
msg315832 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-04-27 10:01
The current discrepancy is odd when you compare it to the equivalent generator expression:

    {k:v for k, v in iterable}

    dict(((k, v) for k, v in iterable))

It would never have occurred to me to expect the evaluation order to match a fully unrolled loop with a nested "d[k] = v" assignment, because the dict constructor doesn't work that way - it accepts an iterable of 2-tuples.

PEP 274 also specifies the iterable-of-2-tuples interpretation (using a list comprehension as its baseline rather than a generator expression): https://www.python.org/dev/peps/pep-0274/#semantics
msg315874 - (view) Author: janzert (janzert) * Date: 2018-04-29 01:15
Just as a note so the email discussion isn't forever lost to the void.

In an unrelated thread on python-dev recently there was a short discussion on this topic in which both Guido van Rossum[1] and Tim Peters[2] gave the opinion that this should change should probably be made.

1: https://mail.python.org/pipermail/python-dev/2018-April/153122.html
2: https://mail.python.org/pipermail/python-dev/2018-April/153134.html
History
Date User Action Args
2018-04-29 01:15:39janzertsetnosy: + janzert
messages: + msg315874
2018-04-27 10:01:37ncoghlansetnosy: + ncoghlan

messages: + msg315832
versions: + Python 3.8, - Python 3.7
2017-02-27 09:35:03rhettingersetnosy: + rhettinger
messages: + msg288623
2017-02-27 08:39:13EvKounissetnosy: + EvKounis
2017-02-26 13:14:41xiang.zhangsetnosy: + xiang.zhang
2017-02-25 19:54:43Jim Fasarakis-Hilliardcreate