Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide optimized HMAC digest #76614

Closed
tiran opened this issue Dec 27, 2017 · 2 comments
Closed

Provide optimized HMAC digest #76614

tiran opened this issue Dec 27, 2017 · 2 comments
Labels
3.7 (EOL) end of life extension-modules C modules in the Modules dir performance Performance or resource usage

Comments

@tiran
Copy link
Member

tiran commented Dec 27, 2017

BPO 32433
Nosy @gpshead, @tiran
PRs
  • bpo-32433: Optimized HMAC digest #5023
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2018-01-27.08:57:12.584>
    created_at = <Date 2017-12-27.19:00:28.290>
    labels = ['extension-modules', '3.7', 'performance']
    title = 'Provide optimized HMAC digest'
    updated_at = <Date 2018-01-27.08:57:12.583>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2018-01-27.08:57:12.583>
    actor = 'christian.heimes'
    assignee = 'none'
    closed = True
    closed_date = <Date 2018-01-27.08:57:12.584>
    closer = 'christian.heimes'
    components = ['Extension Modules']
    creation = <Date 2017-12-27.19:00:28.290>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32433
    keywords = ['patch']
    message_count = 2.0
    messages = ['309097', '310849']
    nosy_count = 2.0
    nosy_names = ['gregory.p.smith', 'christian.heimes']
    pr_nums = ['5023']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'performance'
    url = 'https://bugs.python.org/issue32433'
    versions = ['Python 3.7']

    @tiran
    Copy link
    Member Author

    tiran commented Dec 27, 2017

    hmac.HMAC's flexibility comes with a cost. To create the MAC of a message, it has to create between three and four hash instances (inner, outer, key for long keys, result), multiple bound method objects and other temporary objects. For short messages, memory allocation, object handling and GIL release/acquire operations dominate the performance.

    I propose to provide a fast one-shot HMAC function: hmac.digest(key, msg, digstmod) -> bytes function. A PoC implementation based on OpenSSL's HMAC() function and a pure Python implementation as inlined HMAC showed promising performance improvements. The C implementation is 3 times faster.

    Standard HMAC:
    $ ./python -m timeit -n200000 -s "import hmac" -- "hmac.HMAC(b'key', b'message', 'sha256').digest()"
    200000 loops, best of 5: 5.38 usec per loop

    Optimized Python code:
    $ ./python -m timeit -n 200000 -s "import hmac; hmac._hashopenssl = None" -- "hmac.digest(b'key', b'message', 'sha256')"
    200000 loops, best of 5: 3.87 usec per loop

    OpenSSL HMAC()
    $ ./python -m timeit -n 200000 -s "import hmac" -- "hmac.digest(b'key', b'message', 'sha256')"
    200000 loops, best of 5: 1.82 usec per loop

    @tiran tiran added 3.7 (EOL) end of life extension-modules C modules in the Modules dir performance Performance or resource usage labels Dec 27, 2017
    @tiran
    Copy link
    Member Author

    tiran commented Jan 27, 2018

    New changeset 2f050c7 by Christian Heimes in branch 'master':
    bpo-32433: Optimized HMAC digest (bpo-5023)
    2f050c7

    @tiran tiran closed this as completed Jan 27, 2018
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life extension-modules C modules in the Modules dir performance Performance or resource usage
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant