Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C Fast path for namedtuple's property/itemgetter pair #76673

Closed
rhettinger opened this issue Jan 4, 2018 · 12 comments
Closed

C Fast path for namedtuple's property/itemgetter pair #76673

rhettinger opened this issue Jan 4, 2018 · 12 comments
Labels
3.8 only security fixes extension-modules C modules in the Modules dir

Comments

@rhettinger
Copy link
Contributor

BPO 32492
Nosy @rhettinger, @serhiy-storchaka, @miss-islington
PRs
  • bpo-32492: C Fast path for namedtuple's property/itemgetter pair #5100
  • bpo-32492: 1.6x speed up in namedtuple attribute access using C fast-path #10495
  • bpo-32492: Tweak _collections._tuplegetter. #11367
  • bpo-32492: Tweak _collections._tuplegetter. #11367
  • bpo-32492: Tweak _collections._tuplegetter. #11367
  • bpo-32492: Add missing whatsnew entries for itemgetter and namedtuple #11897
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2018-12-31.12:15:44.489>
    created_at = <Date 2018-01-04.18:05:43.360>
    labels = ['extension-modules', '3.8']
    title = "C Fast path for namedtuple's property/itemgetter pair"
    updated_at = <Date 2019-02-16.20:30:47.658>
    user = 'https://github.com/rhettinger'

    bugs.python.org fields:

    activity = <Date 2019-02-16.20:30:47.658>
    actor = 'ned.deily'
    assignee = 'none'
    closed = True
    closed_date = <Date 2018-12-31.12:15:44.489>
    closer = 'serhiy.storchaka'
    components = ['Extension Modules']
    creation = <Date 2018-01-04.18:05:43.360>
    creator = 'rhettinger'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32492
    keywords = ['patch']
    message_count = 12.0
    messages = ['309477', '309478', '309481', '309486', '311168', '311171', '316822', '332747', '332754', '332755', '332813', '335712']
    nosy_count = 3.0
    nosy_names = ['rhettinger', 'serhiy.storchaka', 'miss-islington']
    pr_nums = ['5100', '10495', '11367', '11367', '11367', '11897']
    priority = None
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue32492'
    versions = ['Python 3.8']

    @rhettinger
    Copy link
    Contributor Author

    Attribute access for named tuples can be made as fast as attribute access of instances of classes with slots. All that is needed is a C subclass of property with it own __get__ that directly look's up the tuple field using PyTuple_GET_ITEM using an index stored in a C struct rather than as an integer object.

    @rhettinger rhettinger added 3.7 (EOL) end of life extension-modules C modules in the Modules dir labels Jan 4, 2018
    @serhiy-storchaka
    Copy link
    Member

    Alternatively we could consider the way of optimizing property or itemgetter (or both). This may be more complex solution, but if the added complexity is not too large, this solution may be more preferable because it will speed up not only namedtuple.

    @rhettinger
    Copy link
    Contributor Author

    Joe Jevnik at already worked through attempts optimize property/itemgetter. There isn't much fruit left there.

    What I've outlined here should be several times faster. The code will end-up being very similar to the __get__ for member objects which is currently the fastest form of attribute access.

    @serhiy-storchaka
    Copy link
    Member

    Ah, right. property already is hardy optimized, and specialized version of itemgetter gives only around 35% of maximal speed up.

    @rhettinger
    Copy link
    Contributor Author

    Ned, I don't have time to finish this before 3.7b1. Will you approve for inclusion in 3.7b2? This isn't a user visible feature. It's goal is to bring named tuple attribute lookup performance up to parity with regular classes using __slots__.

    @ned-deily
    Copy link
    Member

    OK, it would be nice to get this in and the risk seems relatively low. Good luck!

    @ned-deily ned-deily added 3.8 only security fixes deferred-blocker labels Jan 29, 2018
    @ned-deily
    Copy link
    Member

    Deferred to 3.8

    @rhettinger
    Copy link
    Contributor Author

    New changeset 3f5fc70 by Raymond Hettinger (Pablo Galindo) in branch 'master':
    bpo-32492: 1.6x speed up in namedtuple attribute access using C fast-path (bpo-10495)
    3f5fc70

    @serhiy-storchaka
    Copy link
    Member

    There are few post-commit tweaks in PR 11367:

    • Removed the docstrings cache. I have not found significant difference, but this make the code simpler.
    • Improved tests. Merged test_doc_writable and test_namedtuple_can_mutate_doc_of_descriptors_independently in the single test, added tests for immutability, hashing, field descriptors and help(), and rewriting old checks that used eval().
    • Unified names of tp_descr_get and tp_descr_set functions. This will help to search implementations of custom descriptors.

    @serhiy-storchaka
    Copy link
    Member

    See also bpo-35619.

    @serhiy-storchaka
    Copy link
    Member

    New changeset 052b2df by Serhiy Storchaka in branch 'master':
    bpo-32492: Tweak _collections._tuplegetter. (GH-11367)
    052b2df

    @miss-islington
    Copy link
    Contributor

    New changeset 63fa1cf by Miss Islington (bot) (Raymond Hettinger) in branch 'master':
    bpo-32492: Add missing whatsnew entries for itemgetter and namedtuple (GH-11897)
    63fa1cf

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes extension-modules C modules in the Modules dir
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants