classification
Title: Warn against removing elements from a list (or seq) while iterating
Type: behavior Stage: resolved
Components: Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Eklutna, eric.araujo, petri.lehtinen
Priority: normal Keywords:

Created on 2012-06-28 03:44 by Eklutna, last changed 2012-07-02 19:41 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
listRemovalBug.py Eklutna, 2012-06-28 03:44 File contains repro for bug.
Messages (3)
msg164219 - (view) Author: Isaac (Eklutna) Date: 2012-06-28 03:44
The simple repro below, shows that if a list of strings has two consecutive items that begin with the same letter, an iteration over the list to find and remove all strings that start with that letter fails.  The second string that starts with the same letter to remove remains in the list.

In the example below, both "bananna" and "blueberry" should be removed from the list, but only "bananna" is removed.

I verified this on both 2.7 and 3.2.

-----------------------------------------------------
--- Output ---
--------------
Before: ['apple', 'bananna', 'blueberry', 'coconut']
After:  ['apple', 'blueberry', 'coconut']

-----------------------------------------------------
--- Repro ---
-------------
itemList = ["apple", "bananna", "blueberry", "coconut"]
print("Before: {0}".format(itemList))

for item in itemList:
    if(item.startswith("b")):
        itemList.remove(item)

print("After:  {0}".format(itemList))
msg164220 - (view) Author: Petri Lehtinen (petri.lehtinen) * (Python committer) Date: 2012-06-28 05:06
This happens because you modify the list while iterating over it, which makes the loop not work as you expect. Essentially, when you remove the item that's currently being pointed to, the loop skips over the next item.

An idiomatic way to remove items from a list is to use a list comprehension to create a new list without the unwanted items:

item_list = ["apple", "bananna", "blueberry", "coconut"]
new_list = [item for item in item_list if not item.startswith('b')]
msg164536 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2012-07-02 19:41
Maybe there is a section in the documentation that could be enhanced to make readers expect this behavior?  (reference for list or tutorial)
History
Date User Action Args
2012-07-02 19:41:41eric.araujosetnosy: + eric.araujo

messages: + msg164536
title: list.startswith() and list.remove() fails to catch consecutive items in a list. -> Warn against removing elements from a list (or seq) while iterating
2012-06-28 05:06:27petri.lehtinensetstatus: open -> closed

nosy: + petri.lehtinen
messages: + msg164220

resolution: not a bug
stage: resolved
2012-06-28 03:44:53Eklutnacreate