Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance regression for making bound methods #83298

Open
rhettinger opened this issue Dec 21, 2019 · 7 comments
Open

Performance regression for making bound methods #83298

rhettinger opened this issue Dec 21, 2019 · 7 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@rhettinger
Copy link
Contributor

BPO 39117
Nosy @rhettinger, @vstinner, @benjaminp, @methane, @serhiy-storchaka, @pablogsal

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-12-21.20:13:01.396>
labels = ['interpreter-core', '3.9', 'performance']
title = 'Performance regression for making bound methods'
updated_at = <Date 2020-04-28.21:51:46.108>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2020-04-28.21:51:46.108>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2019-12-21.20:13:01.396>
creator = 'rhettinger'
dependencies = []
files = []
hgrepos = []
issue_num = 39117
keywords = []
message_count = 7.0
messages = ['358781', '358802', '358807', '367556', '367566', '367567', '367568']
nosy_count = 6.0
nosy_names = ['rhettinger', 'vstinner', 'benjamin.peterson', 'methane', 'serhiy.storchaka', 'pablogsal']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue39117'
versions = ['Python 3.9']

@rhettinger
Copy link
Contributor Author

$ python3.9 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m'
1000000 loops, best of 11: 230 nsec per loop

$ python3.8 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m'
2000000 loops, best of 11: 149 nsec per loop

$ python3.7 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m'
2000000 loops, best of 11: 159 nsec per loop

$ python3.6 -m timeit -r 11 -s 'class A: pass' -s 'A.m = lambda s: None' -s 'a = A()' 'a.m; a.m; a.m; a.m; a.m'
10000000 loops, best of 11: 0.159 usec per loop

Timings made using the recent released python.org macOS 64-bit builds.

@rhettinger rhettinger added 3.9 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Dec 21, 2019
@benjaminp
Copy link
Contributor

Probably bpo-37340

@methane
Copy link
Member

methane commented Dec 23, 2019

Is this regression is large enough to revive the free_list for bound methods?

@rhettinger
Copy link
Contributor Author

This performance regression in still present in 3.9.0a6

Results from Tools/scripts/var_access_benchmark.py:

                     Python 3.8.2       Python 3.9.0a6
                     ------------       \--------------

read_boundmethod 27.7 ns 47.1 ns

Is this regression is large enough to revive the free_list
for bound methods?

Yes!

@vstinner
Copy link
Member

read_boundmethod 27.7 ns 47.1 ns

Extract of Tools/scripts/var_access_benchmark.py:

def read_boundmethod(trials=trials, a=A()):
    for t in trials:
        a.m;    a.m;    a.m;    a.m;    a.m
        a.m;    a.m;    a.m;    a.m;    a.m
        a.m;    a.m;    a.m;    a.m;    a.m
        a.m;    a.m;    a.m;    a.m;    a.m
        a.m;    a.m;    a.m;    a.m;    a.m

Which kind of code pattern is impacted by this performance regression, apart this micro-benchmark? Do you notice a significant slowdown in pyperformance?

When pyperformance was run before the change was merged, there was no significant difference:
https://bugs.python.org/issue37340#msg348425

In bpo-37340, you wrote that sorted(data, key=str.upper) is 70% slower. Would you mind to provide the benchmark?

@vstinner
Copy link
Member

I compared sorted(data, key=str.upper) performance between before the removal of the free list (parent of commit 3e54b57) and the current master branch, I get:

Mean +- std dev: [master] 167 us +- 4 us -> [cache] 179 us +- 7 us: 1.07x slower (+7%)

I cannot reproduce the "70% slowdown".

Raymond: How did you compile Python? What is your OS? How did you run your benchmark?

---

It's a quick & dirty benchmark run. I didn't use LTO+PGO and I didn't isolate my CPU. Maybe the difference of 12 ns is just caused by the noise of my coarse benchmark procedure.

I used commands:

# master
git checkout master
git clean -fdx
./configure && make && ./python -m venv env && env/bin/python -m pip install pyperf
env/bin/python -m pyperf timeit -s 'import random; data = [f"x{i}" for i in range(1000)]; random.shuffle(data)' 'sorted(data, key=str.upper)' -o ../master.json

# before removal of the free list
git checkout 3e54b57^
git clean -fdx
./configure && make && ./python -m venv env && env/bin/python -m pip install pyperf
env/bin/python -m pyperf timeit -s 'import random; data = [f"x{i}" for i in range(1000)]; random.shuffle(data)' 'sorted(data, key=str.upper)' -o ../master.json

That's a Fedora 31 using GCC -O3 (GCC 9.3.1).

--

The commit before the removal of the free list is:

commit 76b6451 (HEAD)
Date: Fri Jul 26 03:30:33 2019 +0200

Well, maybe I did a mistake.

@vstinner
Copy link
Member

By the way, in sorted(data, key=str.upper): str.upper is an unbound method, no?

$ ./python
Python 3.9.0a6+ (heads/master:d9a43e20fa, Apr 28 2020, 23:50:37) 
>>> type(str.upper)
<class 'method_descriptor'>

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
@iritkatriel iritkatriel added 3.11 only security fixes 3.10 only security fixes 3.12 bugs and security fixes labels Sep 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
Projects
None yet
Development

No branches or pull requests

5 participants