classification
Title: C Fast path for namedtuple's property/itemgetter pair
Type: Stage: resolved
Components: Extension Modules Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, rhettinger, serhiy.storchaka
Priority: Keywords: patch

Created on 2018-01-04 18:05 by rhettinger, last changed 2019-02-16 20:30 by ned.deily. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5100 closed rhettinger, 2018-01-04 18:07
PR 10495 merged pablogsal, 2018-11-13 00:59
PR 11367 merged serhiy.storchaka, 2018-12-30 17:58
PR 11367 merged serhiy.storchaka, 2018-12-30 17:58
PR 11367 merged serhiy.storchaka, 2018-12-30 17:58
PR 11897 merged rhettinger, 2019-02-16 19:56
Messages (12)
msg309477 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-01-04 18:05
Attribute access for named tuples can be made as fast as attribute access of instances of classes with slots.  All that is needed is a C subclass of property with it own __get__ that directly look's up the tuple field using PyTuple_GET_ITEM using an index stored in a C struct rather than as an integer object.
msg309478 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-01-04 18:39
Alternatively we could consider the way of optimizing property or itemgetter (or both). This may be more complex solution, but if the added complexity is not too large, this solution may be more preferable because it will speed up not only namedtuple.
msg309481 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-01-04 20:28
Joe Jevnik at already worked through attempts optimize property/itemgetter.   There isn't much fruit left there.

What I've outlined here should be several times faster.  The code will end-up being very similar to the __get__ for member objects which is currently the fastest form of attribute access.
msg309486 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-01-04 22:31
Ah, right. property already is hardy optimized, and specialized version of itemgetter gives only around 35% of maximal speed up.
msg311168 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-01-29 20:42
Ned, I don't have time to finish this before 3.7b1.  Will you approve for inclusion in 3.7b2?  This isn't a user visible feature.  It's goal is to bring named tuple attribute lookup performance up to parity with regular classes using __slots__.
msg311171 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-01-29 21:02
OK, it would be nice to get this in and the risk seems relatively low.  Good luck!
msg316822 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-05-16 15:43
Deferred to 3.8
msg332747 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-12-30 09:24
New changeset 3f5fc70c6213008243e7d605f7d8a2d8f94cf919 by Raymond Hettinger (Pablo Galindo) in branch 'master':
bpo-32492: 1.6x speed up in namedtuple attribute access using C fast-path (#10495)
https://github.com/python/cpython/commit/3f5fc70c6213008243e7d605f7d8a2d8f94cf919
msg332754 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-30 18:05
There are few post-commit tweaks in PR 11367:

* Removed the docstrings cache. I have not found significant difference, but this make the code simpler.
* Improved tests. Merged test_doc_writable and test_namedtuple_can_mutate_doc_of_descriptors_independently in the single test, added tests for immutability, hashing, field descriptors and help(), and rewriting old checks that used eval().
* Unified names of tp_descr_get and tp_descr_set functions. This will help to search implementations of custom descriptors.
msg332755 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-30 18:06
See also issue35619.
msg332813 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-12-31 12:15
New changeset 052b2dfdc967a8c061ff9561534e905009b88b8c by Serhiy Storchaka in branch 'master':
bpo-32492: Tweak _collections._tuplegetter. (GH-11367)
https://github.com/python/cpython/commit/052b2dfdc967a8c061ff9561534e905009b88b8c
msg335712 - (view) Author: miss-islington (miss-islington) Date: 2019-02-16 20:02
New changeset 63fa1cfece4912110ce3a0ff11fb3ade3ff5e756 by Miss Islington (bot) (Raymond Hettinger) in branch 'master':
bpo-32492: Add missing whatsnew entries for itemgetter and namedtuple (GH-11897)
https://github.com/python/cpython/commit/63fa1cfece4912110ce3a0ff11fb3ade3ff5e756
History
Date User Action Args
2019-02-16 20:30:47ned.deilysetnosy: - ned.deily
2019-02-16 20:02:26miss-islingtonsetnosy: + miss-islington
messages: + msg335712
2019-02-16 19:56:56rhettingersetpull_requests: + pull_request11924
2018-12-31 12:15:44serhiy.storchakasetstatus: open -> closed
2018-12-31 12:15:21serhiy.storchakasetmessages: + msg332813
2018-12-30 18:06:10serhiy.storchakasetmessages: + msg332755
2018-12-30 18:05:11serhiy.storchakasetstatus: closed -> open

messages: + msg332754
2018-12-30 17:58:51serhiy.storchakasetpull_requests: + pull_request10703
2018-12-30 17:58:45serhiy.storchakasetpull_requests: + pull_request10702
2018-12-30 17:58:39serhiy.storchakasetpull_requests: + pull_request10701
2018-12-30 09:24:55rhettingersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-12-30 09:24:13rhettingersetmessages: + msg332747
2018-11-13 01:01:28pablogsalsetpull_requests: - pull_request9755
2018-11-13 01:01:23pablogsalsetpull_requests: - pull_request9756
2018-11-13 00:59:59pablogsalsetpull_requests: + pull_request9756
2018-11-13 00:59:59pablogsalsetpull_requests: + pull_request9755
2018-11-13 00:59:55pablogsalsetpull_requests: + pull_request9754
2018-05-16 15:43:16ned.deilysetpriority: deferred blocker ->

messages: + msg316822
versions: - Python 3.7
2018-01-29 21:02:17ned.deilysetpriority: normal -> deferred blocker

messages: + msg311171
versions: + Python 3.8
2018-01-29 20:42:29rhettingersetnosy: + ned.deily
messages: + msg311168
2018-01-04 22:31:04serhiy.storchakasetmessages: + msg309486
2018-01-04 20:28:55rhettingersetmessages: + msg309481
2018-01-04 18:39:47serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg309478
2018-01-04 18:07:23rhettingersetkeywords: + patch
stage: patch review
pull_requests: + pull_request4968
2018-01-04 18:05:43rhettingercreate