classification
Title: Intersection of dict view with iterator returns empty set
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: corona10, lukasz.langa, rhettinger, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2019-09-18 07:10 by serhiy.storchaka, last changed 2021-07-15 04:01 by corona10. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16602 merged corona10, 2019-10-06 06:21
PR 16611 merged vstinner, 2019-10-07 10:05
Messages (8)
msg352705 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-09-18 07:10
In 3.8:

>>> {1: 2, 3: 4}.keys() & {1, 2}
{1}

In master:

>>> {1: 2, 3: 4}.keys() & iter([1, 2])
set()

The behavior was changed in issue27575.
msg354033 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-10-06 11:28
New changeset c38e725d17537b20ff090b1b5ec7db1820ff9b63 by Serhiy Storchaka (Dong-hee Na) in branch 'master':
bpo-38210: Fix intersection operation with dict view and iterator. (GH-16602)
https://github.com/python/cpython/commit/c38e725d17537b20ff090b1b5ec7db1820ff9b63
msg354034 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-10-06 11:29
Thank you Dong-hee for your fix!
msg354045 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-10-06 19:37
Please add a test for this regression.
msg354047 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-10-06 19:51
It was added. Do you mean any special?
msg354072 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-10-07 10:28
New changeset d97f1ce6dba6c4aa5614adc06ad2e0948709845c by Victor Stinner in branch 'master':
bpo-38210: Fix compiler warning in dictobject.c (GH-16611)
https://github.com/python/cpython/commit/d97f1ce6dba6c4aa5614adc06ad2e0948709845c
msg397494 - (view) Author: Ɓukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-14 16:24
This caused an unintentional behavior change in the following code:

>>> {1: 2}.items() & {1: {2: 3}}.items()
set()

Before this change, Python 3.6 - 3.8 behaved like this instead:

>>> {1: 2}.items() & {1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'

Interestingly, this doesn't seem to have a negative effect on correctness as the silently omitted unhashable (k, v) pair is only omitted if it's different between the two dictionaries:

>>> {1: {2: 4}}.items() & {1: {2: 3}}.items()
set()
>>> {2: 1, 1: {2: 4}}.items() & {2: 1, 1: {2: 3}}.items()
{(2, 1)}

If it's the same, we still get an error in Python 3.9:

>>> {1: {2: 3}}.items() & {1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'
>>> {2: 1, 1: {2: 3}}.items() & {2: 1, 1: {2: 3}}.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'dict'
msg397523 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2021-07-15 04:01
>Interestingly, this doesn't seem to have a negative effect on correctness as the silently omitted unhashable

I think so too.

The error actually raises when adding the object into the set.
https://github.com/python/cpython/blob/818628c2da99ba0376313971816d472c65c9a9fc/Objects/dictobject.c#L4384

Since the target object to be added is dynamically generated, I think that the issue does not need to be fixed.
Otherwise, we have to check that all objects are addable to `set` object before executing this operation but it looks harmful to performance.
History
Date User Action Args
2021-07-15 04:01:41corona10setmessages: + msg397523
2021-07-14 16:24:13lukasz.langasetnosy: + lukasz.langa
messages: + msg397494
2019-10-07 10:28:19vstinnersetnosy: + vstinner
messages: + msg354072
2019-10-07 10:05:29vstinnersetpull_requests: + pull_request16199
2019-10-06 19:51:25serhiy.storchakasetmessages: + msg354047
2019-10-06 19:37:20rhettingersetmessages: + msg354045
2019-10-06 11:29:41serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg354034

stage: patch review -> resolved
2019-10-06 11:28:39serhiy.storchakasetmessages: + msg354033
2019-10-06 09:21:20corona10setnosy: + corona10
2019-10-06 06:21:50corona10setkeywords: + patch
stage: patch review
pull_requests: + pull_request16190
2019-09-18 07:10:06serhiy.storchakacreate