This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ronaldoussoren
Recipients docs@python, ezio.melotti, marco.buttu, r.david.murray, ronaldoussoren
Date 2013-07-10.15:33:08
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1373470388.55.0.429274407892.issue18424@psf.upfronthosting.co.za>
In-reply-to
Content
Appending a sequence of lists with sum is inefficient because it (currently) does a lot of copying, and that gets noticable when you sum a larger number of lists

Note how using sum for add 200 lists is more than twice as long as adding 100 lists:

ronald@gondolin[0]$ python -m timeit -s "lists=[['a']*100 for i in range(100)]" "sum(lists, [])"
100 loops, best of 3: 2.04 msec per loop


ronald@gondolin[0]$ python -m timeit -s "lists=[['a']*100 for i in range(200)]" "sum(lists, [])"
100 loops, best of 3: 9.2 msec per loop


Also note how using itertools.chain is both a lot faster and behaves better:

ronald@gondolin[0]$ python -m timeit -s "import itertools; lists=[['a']*100 for i in range(100)]" "list(itertools.chain.from_iterable(lists))"
10000 loops, best of 3: 165 usec per loop


ronald@gondolin[0]$ python -m timeit -s "import itertools; lists=[['a']*100 for i in range(100)]" "list(itertools.chain.from_iterable(lists))"
10000 loops, best of 3: 155 usec per loop

(I used python2.7 for this, the same behavior can be seem with python 3).

See also #18305, which proposed a small change to how sum works which would fix the performance problems for summing a sequence of lists (before going too far and proposing to add special-case tuples and string)
History
Date User Action Args
2013-07-10 15:33:08ronaldoussorensetrecipients: + ronaldoussoren, ezio.melotti, r.david.murray, docs@python, marco.buttu
2013-07-10 15:33:08ronaldoussorensetmessageid: <1373470388.55.0.429274407892.issue18424@psf.upfronthosting.co.za>
2013-07-10 15:33:08ronaldoussorenlinkissue18424 messages
2013-07-10 15:33:08ronaldoussorencreate