This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Remove backwards compatibility in _heapq for performance
Type: performance Stage:
Components: Extension Modules Versions: Python 2.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: kristjan.jonsson, rhettinger
Priority: normal Keywords: easy, patch

Created on 2008-10-26 11:53 by kristjan.jonsson, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
heapq1.patch kristjan.jonsson, 2008-10-26 11:53
Messages (3)
msg75224 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2008-10-26 11:53
Comparing _heapq with our own legacy C implementation, blue.heapq at 
CCP, I noticed that ours was somewhat faster.
I discovered that a lot of effort is being spent to dynamically search 
for a __lt__ operator, to provide backwards compatibility.  I think we 
should consider dropping that after this much time, especially for a 
new python version.  Running this code:

from timeit import *
s = """
import heapq
import random
l = [random.random() for i in xrange(10000)]
"""

print "heapify", Timer("heapq.heapify(list(l))", s).timeit(100)

s = s + """
heapq.heapify(l)
"""
print "pushpop", Timer("heapq.heappushpop(l,random.random())", s).timeit
(500000)


would give 
heapify 0.289102944648
pushpop 0.343299078514
before.  After the patch, we get:
heapify 0.0958507994731
pushpop 0.220800967721

This is "release" code on a snappy windows machine.
msg75225 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2008-10-26 13:07
The compatability code was just added in Py2.6 and is needed for apps
like Twisted that currently rely on __le__ being tested.  In 3.0, the
compatability code is removed and full speed is restored.

Also, the timing suite exaggerates the effect.  A more typical use of
heaps involves a heap of tuples with the first tuple element being used
as a priority level.  That increases the comparison time and decreases
the relative significance of the dispatch logic.
msg75226 - (view) Author: Kristján Valur Jónsson (kristjan.jonsson) * (Python committer) Date: 2008-10-26 13:49
I am sorry for not doing my research about the age of the compatibility 
fix.
However, modifying the test slightly to work with tuples of
(random.random(), random.random())
shows a performance increase from:

heapify 0.366187741738
pushpop 0.541365033824
replace 2.69348946584
push and pop 3.12545093022

to:

heapify 0.186918030085
pushpop 0.405662172148
replace 1.46039447751
push and pop 1.75253663524

This does look like a large price to pay for this compatibility layer.
History
Date User Action Args
2022-04-11 14:56:40adminsetgithub: 48457
2008-10-26 13:49:27kristjan.jonssonsetkeywords: patch, patch, easy
messages: + msg75226
2008-10-26 13:07:45rhettingersetstatus: open -> closed
nosy: + rhettinger
messages: + msg75225
assignee: rhettinger
keywords: patch, patch, easy
resolution: rejected
2008-10-26 11:53:06kristjan.jonssoncreate