This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients alexandre.vassalotti, pitrou, rhettinger, serhiy.storchaka
Date 2016-03-06.14:57:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1457276257.9.0.371160640509.issue25776@psf.upfronthosting.co.za>
In-reply-to
Content
There is the extra special case code for bytes and bytearray iterators and reversed list iterator. The patch just extends the optimizations to other builtin iterators.

In the example with itertools.islice() the overhead of pickling redundant data is 20%. If there are a lot of iterators, the overhead can be up to 90%.

>>> import pickle, pickletools, itertools
>>> len(pickletools.optimize(pickle.dumps([itertools.islice('abcdefgh', 4) for i in range(1000)], 4)))
Unpatched: 23059
Patched:   12059

Of course this is degenerated case. But if the data contains a lot of initial iterators of the same sequence or of short sequences, or exhausted iterators, the benefit can be not small.

Yet one benefit from the patch is that it speeds up copying and deepcopying initial and exhausted iterators.

$ ./python -m timeit -s "from itertools import islice; from copy import copy; it = islice('abcdefgh', 4)" -- "copy(it)"
Unpatched: 7.37 usec per loop
Patched:   6.4 usec per loop

$ ./python -m timeit -s "from itertools import islice; from copy import deepcopy; it = islice('abcdefgh', 4)" -- "deepcopy(it)"
Unpatched: 41.7 usec per loop
Patched:   32.6 usec per loop

$ ./python -m timeit -s "from copy import copy; it = iter('abcdefgh')" -- "copy(it)"
Unpatched: 10.4 usec per loop
Patched:   9.67 usec per loop

$ ./python -m timeit -s "from copy import deepcopy; it = iter('abcdefgh')" -- "deepcopy(it)"
Unpatched: 21.1 usec per loop
Patched:   18.3 usec per loop

$ ./python -m timeit -s "from copy import copy; it = iter(list('abcdefgh'))" -- "copy(it)"
Unpatched: 10.3 usec per loop
Patched:   9.54 usec per loop

$ ./python -m timeit -s "from copy import deepcopy; it = iter(list('abcdefgh'))" -- "deepcopy(it)"
Unpatched: 39.7 usec per loop
Patched:   36.8 usec per loop
History
Date User Action Args
2016-03-06 14:57:37serhiy.storchakasetrecipients: + serhiy.storchaka, rhettinger, pitrou, alexandre.vassalotti
2016-03-06 14:57:37serhiy.storchakasetmessageid: <1457276257.9.0.371160640509.issue25776@psf.upfronthosting.co.za>
2016-03-06 14:57:37serhiy.storchakalinkissue25776 messages
2016-03-06 14:57:37serhiy.storchakacreate