Message255699
Proposed patch makes a number of classes produce more compact pickle data in common case. This includes iterators of list, tuple, str, bytes, bytearray, enumerate, array, deque, iterator object for classes with __getitem__, some itertools iterators, and non-iterator objects: slice, bytearray, deque. This is achieved by omitting default constructor arguments or state.
Exhausted iterators are pickled as iter(()). This is not new, exhausted bytes, and bytearray iterators, and reversed list iterator are already pickled as iter('') or iter([]) correspondingly. iter(()) is just the simplest way to create an empty iterator and it has the most compact pickle representation.
An example.
Unpatched:
>>> import pickle, pickletools, itertools
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
80
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
66
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
0: \x80 PROTO 3
2: c GLOBAL 'itertools islice'
20: ( MARK
21: c GLOBAL 'builtins iter'
36: X BINUNICODE 'abcdefgh'
49: \x85 TUPLE1
50: R REDUCE
51: K BININT1 0
53: b BUILD
54: K BININT1 0
56: K BININT1 4
58: K BININT1 1
60: t TUPLE (MARK at 20)
61: R REDUCE
62: K BININT1 0
64: b BUILD
65: . STOP
highest protocol among opcodes = 2
Patched:
>>> len(pickle.dumps(itertools.islice('abcdefgh', 4), 3))
69
>>> len(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
55
>>> pickletools.dis(pickletools.optimize(pickle.dumps(itertools.islice('abcdefgh', 4), 3)))
0: \x80 PROTO 3
2: c GLOBAL 'itertools islice'
20: c GLOBAL 'builtins iter'
35: X BINUNICODE 'abcdefgh'
48: \x85 TUPLE1
49: R REDUCE
50: K BININT1 4
52: \x86 TUPLE2
53: R REDUCE
54: . STOP
highest protocol among opcodes = 2 |
|
Date |
User |
Action |
Args |
2015-12-02 10:29:16 | serhiy.storchaka | set | recipients:
+ serhiy.storchaka, rhettinger, pitrou, alexandre.vassalotti |
2015-12-02 10:29:15 | serhiy.storchaka | set | messageid: <1449052155.68.0.447852149746.issue25776@psf.upfronthosting.co.za> |
2015-12-02 10:29:15 | serhiy.storchaka | link | issue25776 messages |
2015-12-02 10:29:14 | serhiy.storchaka | create | |
|