Title: add __contains__ for list_iterator (and others) for better performance
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.7
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, rhettinger, serhiy.storchaka, sir-sigurd, terry.reedy
Priority: normal Keywords:

Created on 2017-08-02 18:53 by sir-sigurd, last changed 2017-08-31 07:07 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2988 closed sir-sigurd, 2017-08-02 18:54
Messages (6)
msg299666 - (view) Author: Sergey Fedoseev (sir-sigurd) * Date: 2017-08-02 18:53
> python -mtimeit -s "l = list(range(100000))" "l[-1] in l"
1000 loops, best of 3: 1.34 msec per loop
> python -mtimeit -s "l = list(range(100000))" "l[-1] in iter(l)"                                
1000 loops, best of 3: 1.59 msec per loop
msg299686 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-03 06:32
The patch adds almost 40 line of the code and increases the performance of not well famous feature at best by 10-20%. Adding an optimization for every new iterator type will add a comparable quantity of the code. I think this is too high cost.

Using a common template implementation for iterators (issue27438) would decrease the relative cost of this feature.
msg299773 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-08-05 04:38
Terry, this proposition doesn't change the behavior. It moves the iterator forward during searching. The only effect is inlining __next__ in a loop and getting rid of the overhead of few indirections and calls.
msg299774 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-05 04:56
Sorry, I mistakenly assumed, without carefully checking the C code, that the speedup was from checking the underlying collection, without advancing the iterator.  I presume that " ++it->it_index;" is the statement to the contrary. I should have either asked or found this sooner.  I unlinked the noisy comment.
msg301022 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-30 11:46
I don't think I've ever used `in` on an iterator.  I didn't even expect it to work, and would not consider its use a good practice.
msg301040 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-08-31 07:03
I recommend rejecting this proposal
Date User Action Args
2017-08-31 07:07:26serhiy.storchakasetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2017-08-31 07:03:58rhettingersetmessages: + msg301040
2017-08-30 11:46:50pitrousetnosy: + pitrou
messages: + msg301022
2017-08-05 04:56:40terry.reedysetmessages: + msg299774
2017-08-05 04:51:54terry.reedysetmessages: - msg299770
2017-08-05 04:38:34serhiy.storchakasetmessages: + msg299773
2017-08-05 04:08:27terry.reedysettype: performance -> enhancement

messages: + msg299770
nosy: + terry.reedy
2017-08-03 06:32:08serhiy.storchakasetversions: + Python 3.7
nosy: + rhettinger, serhiy.storchaka

messages: + msg299686

components: + Interpreter Core
stage: patch review
2017-08-02 18:54:38sir-sigurdsetpull_requests: + pull_request3025
2017-08-02 18:53:46sir-sigurdcreate