This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients alexandre.vassalotti, pitrou, rhettinger, serhiy.storchaka
Date 2015-12-02.10:29:13
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1449052155.68.0.447852149746.issue25776@psf.upfronthosting.co.za>
In-reply-to
Content
Proposed patch makes a number of classes produce more compact pickle data in common case. This includes iterators of list, tuple, str, bytes, bytearray, enumerate, array, deque, iterator object for classes with __getitem__, some itertools iterators, and non-iterator objects: slice, bytearray, deque. This is achieved by omitting default constructor arguments or state.

Exhausted iterators are pickled as iter(()). This is not new, exhausted bytes, and bytearray iterators, and reversed list iterator are already pickled as iter('') or iter([]) correspondingly. iter(()) is just the simplest way to create an empty iterator and it has the most compact pickle representation.

An example.

Unpatched:
>>> import pickle, pickletools, itertools
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
80
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
66
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
    0: \x80 PROTO      3
    2: c    GLOBAL     'itertools islice'
   20: (    MARK
   21: c        GLOBAL     'builtins iter'
   36: X        BINUNICODE 'abcdefgh'
   49: \x85     TUPLE1
   50: R        REDUCE
   51: K        BININT1    0
   53: b        BUILD
   54: K        BININT1    0
   56: K        BININT1    4
   58: K        BININT1    1
   60: t        TUPLE      (MARK at 20)
   61: R    REDUCE
   62: K    BININT1    0
   64: b    BUILD
   65: .    STOP
highest protocol among opcodes = 2

Patched:
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
69
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
55
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
    0: \x80 PROTO      3
    2: c    GLOBAL     'itertools islice'
   20: c    GLOBAL     'builtins iter'
   35: X    BINUNICODE 'abcdefgh'
   48: \x85 TUPLE1
   49: R    REDUCE
   50: K    BININT1    4
   52: \x86 TUPLE2
   53: R    REDUCE
   54: .    STOP
highest protocol among opcodes = 2
History
Date User Action Args
2015-12-02 10:29:16serhiy.storchakasetrecipients: + serhiy.storchaka, rhettinger, pitrou, alexandre.vassalotti
2015-12-02 10:29:15serhiy.storchakasetmessageid: <1449052155.68.0.447852149746.issue25776@psf.upfronthosting.co.za>
2015-12-02 10:29:15serhiy.storchakalinkissue25776 messages
2015-12-02 10:29:14serhiy.storchakacreate