This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: dict.fromkeys overallocates when given a sparse dict
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: methane, python-dev, serhiy.storchaka, villemoes
Priority: normal Keywords: patch

Created on 2016-12-19 23:29 by villemoes, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
fromkeys.patch villemoes, 2016-12-19 23:29
fromkeys2.patch methane, 2016-12-20 04:36 review
Messages (5)
msg283654 - (view) Author: Rasmus Villemoes (villemoes) Date: 2016-12-19 23:29
Somewhat related to #23971, but with the opposite sign: Using Py_SIZE on a PyDictObject gives the ma_fill member in 2.7 (and until 3.2 AFAICT), not the ma_used member. So calling dict.fromkeys(d) overallocates significantly if a lot of elements have been deleted from d.

The fix is trivial, though one could of course avoid the function call and do a cast. Not sure this qualifies for 2.7, though.
msg283661 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2016-12-20 03:23
Since `seq` is checked by PyDict_CheckExact(seq) already,
((PyDictObject*)seq)->ma_used may be better.
msg283665 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2016-12-20 04:36
While the patch is trivial, I need LGTM from another comitter.
msg283669 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-20 05:28
LGTM.
msg283672 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-12-20 07:07
New changeset 46e329ef13ae by INADA Naoki in branch '2.7':
Issue #29019: Fix dict.fromkeys(x) overallocates when x is sparce dict.
https://hg.python.org/cpython/rev/46e329ef13ae
History
Date User Action Args
2022-04-11 14:58:40adminsetgithub: 73205
2016-12-20 07:08:22methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-12-20 07:07:40python-devsetnosy: + python-dev
messages: + msg283672
2016-12-20 05:28:12serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg283669
2016-12-20 04:36:23methanesetfiles: + fromkeys2.patch

messages: + msg283665
stage: patch review
2016-12-20 03:23:49methanesetmessages: + msg283661
2016-12-20 03:07:09xiang.zhangsetnosy: + methane
2016-12-19 23:29:42villemoescreate