This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Remove smalltable from set objects
Type: performance Stage:
Components: Interpreter Core Versions: Python 3.5
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: BreamoreBoy, rhettinger
Priority: low Keywords: patch

Created on 2014-12-23 23:40 by rhettinger, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
no_smalltable2.diff rhettinger, 2014-12-23 23:40 Remove smalltable (still need fix for test_sys.py) review
Messages (3)
msg233068 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2014-12-23 23:40
This tracker item is here to record my efforts to re-evaluate whether we were getting much if any benefit from the smalltable in set objects.

Removing the smalltable special case and instead using a memory allocation had the following effects:

* Nice simplification of the code, greatly improving the clarity of the functions for resizing, clearing, and swapping.

* Reduced the memory consumption for sets that were already using memory allocated tables (saved the memory cost of the unused smalltable).

* Nearly doubled the time to allocate and free set objects (see timings below for CLang and GCC).

As a percentage change, the time penalty seems like a killer, but then we're talking about only 1/10th of a μsec per set.



# CLANG #########

~/base_cp/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 39.1 msec per loop
~/cpython/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 76.7 msec per loop
~/base_cp/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 38.8 msec per loop
~/cpython/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 76.6 msec per loop

~/base_cp/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.0964 usec per loop
~/cpython/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.148 usec per loop
~/base_cp/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.0964 usec per loop
~/cpython/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.149 usec per loop

# GCC-4.9 ########
~/base_cp $ ~/base_cp/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.0701 usec per loop
~/base_cp $ ~/cpython/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.155 usec per loop
~/base_cp $ ~/base_cp/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.0691 usec per loop
~/base_cp $ ~/cpython/python.exe -m timeit '{1,2,3}'
10000000 loops, best of 3: 0.157 usec per loop

~/base_cp $ ~/base_cp/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 34.6 msec per loop
~/base_cp $ ~/cpython/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 77 msec per loop
~/base_cp $ ~/base_cp/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 34.1 msec per loop
~/base_cp $ ~/cpython/python.exe -m timeit -s 'from itertools import repeat, starmap' 'list(starmap(set, repeat((), 250000)))'
10 loops, best of 3: 77.3 msec per loop
msg238830 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-03-21 19:44
My feeling is that this is worth doing for the code clarity alone but what do others think about it?
msg281428 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-11-22 01:30
Based on the performance hit, I am retracting this.
History
Date User Action Args
2022-04-11 14:58:11adminsetgithub: 67295
2016-11-22 01:30:01rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg281428
2015-03-21 19:44:58BreamoreBoysetnosy: + BreamoreBoy
messages: + msg238830
2014-12-23 23:40:45rhettingercreate