classification
Title: PyObject_LengthHint is slow
Type: performance Stage: resolved
Components: Interpreter Core Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: benjamin.peterson, pitrou, python-dev, rhettinger, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2013-10-23 19:38 by pitrou, last changed 2013-10-24 20:21 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
lengthhint.patch pitrou, 2013-10-23 19:40 review
lengthhint_2.patch serhiy.storchaka, 2013-10-23 21:43 review
Messages (7)
msg201060 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-23 19:38
PyObject_LengthHint is in the critical patch for many operations (such as list constructor, or list.extend), but it's quite unoptimized.
msg201061 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-23 19:40
Here is a patch. Micro-benchmark:

$ ./python -m timeit -s "l=[]; it=(x for x in ())" "l.extend(it)"
-> before: 0.449 usec per loop
-> after: 0.179 usec per loop
msg201066 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-23 20:23
What if use _PyObject_HasLen?

if (_PyObject_HasLen(o)) {
    res = PyObject_Length(o);
    ...
}

hint = _PyObject_LookupSpecial(o, &PyId___length_hint__);
...
msg201071 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-10-23 21:43
Here is a patch which uses _PyObject_HasLen. It has same performance but is much simpler.
msg201073 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-23 21:49
> Here is a patch which uses _PyObject_HasLen. It has same performance
> but is much simpler.

Thanks!
msg201155 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-10-24 16:33
> Here is a patch which uses _PyObject_HasLen. It has same performance
> but is much simpler.

+1 This will be a nice improvement.
msg201181 - (view) Author: Roundup Robot (python-dev) Date: 2013-10-24 20:20
New changeset bffb49efc383 by Serhiy Storchaka in branch 'default':
Issue #19369: Optimized the usage of __length_hint__().
http://hg.python.org/cpython/rev/bffb49efc383
History
Date User Action Args
2013-10-24 20:21:55serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: resolved
2013-10-24 20:20:18python-devsetnosy: + python-dev
messages: + msg201181
2013-10-24 20:14:58serhiy.storchakasetassignee: serhiy.storchaka
2013-10-24 16:33:17rhettingersetmessages: + msg201155
2013-10-23 21:49:30pitrousetmessages: + msg201073
2013-10-23 21:43:58serhiy.storchakasetfiles: + lengthhint_2.patch

messages: + msg201071
2013-10-23 20:23:45serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg201066
2013-10-23 19:40:47pitrousetfiles: + lengthhint.patch
keywords: + patch
messages: + msg201061
2013-10-23 19:38:27pitroucreate