classification
Title: Speed up comparison of bytes and bytearray object with objects of different types
Type: performance Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: methane, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2020-11-22 12:31 by serhiy.storchaka, last changed 2020-11-22 20:35 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23461 merged serhiy.storchaka, 2020-11-22 12:38
Messages (2)
msg381612 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-22 12:31
The proposed PR speeds up comparison of bytes and bytearray object with objects of different types, especially if use the -b option.

Before:

$ ./python -m timeit -s "x = b''" "x == None"
10000000 loops, best of 5: 29.5 nsec per loop
$ ./python -b -m timeit -s "x = b''" "x == None"
2000000 loops, best of 5: 139 nsec per loop
$ ./python -m timeit -s "x = bytearray()" "x == None"
1000000 loops, best of 5: 282 nsec per loop
$ ./python -b -m timeit -s "x = bytearray()" "x == None"
1000000 loops, best of 5: 282 nsec per loop

After:

$ ./python -m timeit -s "x = b''" "x == None"
10000000 loops, best of 5: 29.7 nsec per loop
$ ./python -b -m timeit -s "x = b''" "x == None"
10000000 loops, best of 5: 29.9 nsec per loop
$ ./python -m timeit -s "x = bytearray()" "x == None"
10000000 loops, best of 5: 32.1 nsec per loop
$ ./python -b -m timeit -s "x = bytearray()" "x == None"
10000000 loops, best of 5: 32.2 nsec per loop

There were two causes of the slowdown:

1. When checked for bytes warning, the code used slow PyObject_IsInstance() which checks the __class__ attribute and looks up several other attributes. Using fast PyUnicode_Check() and PyLong_Check() is enough, because the only case when these methods give different result if you compare with an instance of special class with the __class__ property which return str or int. It is very uncommon case.

2. For bytearray, it tried to get buffers of arguments, and if they did not support the buffer protocol, a TypeError was raised and immediately suppressed. Using fast check PyObject_CheckBuffer() allows to get rid of raising an exception.

Also, PyUnicode_Check() and PyLong_Check() are more convenient, because they do not need error handling.
msg381630 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-22 20:01
New changeset 313467efdc23a1ec8662b77d2001642be598f80a by Serhiy Storchaka in branch 'master':
bpo-42435: Speed up comparison of bytes and bytearray object (GH--23461)
https://github.com/python/cpython/commit/313467efdc23a1ec8662b77d2001642be598f80a
History
Date User Action Args
2020-11-22 20:35:41serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-11-22 20:01:00serhiy.storchakasetmessages: + msg381630
2020-11-22 12:38:28serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request22353
2020-11-22 12:33:48serhiy.storchakasettitle: Spped up comparison of bytes and bytearray object with objects of different types -> Speed up comparison of bytes and bytearray object with objects of different types
2020-11-22 12:31:52serhiy.storchakacreate