Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up comparison of bytes and bytearray object with objects of different types #86601

Closed
serhiy-storchaka opened this issue Nov 22, 2020 · 2 comments
Labels
3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@serhiy-storchaka
Copy link
Member

BPO 42435
Nosy @methane, @serhiy-storchaka
PRs
  • bpo-42435: Speed up comparison of bytes and bytearray object #23461
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-11-22.20:35:41.645>
    created_at = <Date 2020-11-22.12:31:52.316>
    labels = ['interpreter-core', '3.10', 'performance']
    title = 'Speed up comparison of bytes and bytearray object with objects of different types'
    updated_at = <Date 2020-11-22.20:35:41.644>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2020-11-22.20:35:41.644>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-11-22.20:35:41.645>
    closer = 'serhiy.storchaka'
    components = ['Interpreter Core']
    creation = <Date 2020-11-22.12:31:52.316>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 42435
    keywords = ['patch']
    message_count = 2.0
    messages = ['381612', '381630']
    nosy_count = 2.0
    nosy_names = ['methane', 'serhiy.storchaka']
    pr_nums = ['23461']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'performance'
    url = 'https://bugs.python.org/issue42435'
    versions = ['Python 3.10']

    @serhiy-storchaka
    Copy link
    Member Author

    The proposed PR speeds up comparison of bytes and bytearray object with objects of different types, especially if use the -b option.

    Before:

    $ ./python -m timeit -s "x = b''" "x == None"
    10000000 loops, best of 5: 29.5 nsec per loop
    $ ./python -b -m timeit -s "x = b''" "x == None"
    2000000 loops, best of 5: 139 nsec per loop
    $ ./python -m timeit -s "x = bytearray()" "x == None"
    1000000 loops, best of 5: 282 nsec per loop
    $ ./python -b -m timeit -s "x = bytearray()" "x == None"
    1000000 loops, best of 5: 282 nsec per loop

    After:

    $ ./python -m timeit -s "x = b''" "x == None"
    10000000 loops, best of 5: 29.7 nsec per loop
    $ ./python -b -m timeit -s "x = b''" "x == None"
    10000000 loops, best of 5: 29.9 nsec per loop
    $ ./python -m timeit -s "x = bytearray()" "x == None"
    10000000 loops, best of 5: 32.1 nsec per loop
    $ ./python -b -m timeit -s "x = bytearray()" "x == None"
    10000000 loops, best of 5: 32.2 nsec per loop

    There were two causes of the slowdown:

    1. When checked for bytes warning, the code used slow PyObject_IsInstance() which checks the __class__ attribute and looks up several other attributes. Using fast PyUnicode_Check() and PyLong_Check() is enough, because the only case when these methods give different result if you compare with an instance of special class with the __class__ property which return str or int. It is very uncommon case.

    2. For bytearray, it tried to get buffers of arguments, and if they did not support the buffer protocol, a TypeError was raised and immediately suppressed. Using fast check PyObject_CheckBuffer() allows to get rid of raising an exception.

    Also, PyUnicode_Check() and PyLong_Check() are more convenient, because they do not need error handling.

    @serhiy-storchaka serhiy-storchaka added 3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Nov 22, 2020
    @serhiy-storchaka serhiy-storchaka changed the title Spped up comparison of bytes and bytearray object with objects of different types Speed up comparison of bytes and bytearray object with objects of different types Nov 22, 2020
    @serhiy-storchaka serhiy-storchaka changed the title Spped up comparison of bytes and bytearray object with objects of different types Speed up comparison of bytes and bytearray object with objects of different types Nov 22, 2020
    @serhiy-storchaka
    Copy link
    Member Author

    New changeset 313467e by Serhiy Storchaka in branch 'master':
    bpo-42435: Speed up comparison of bytes and bytearray object (GH--23461)
    313467e

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant