This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients alexandre.vassalotti, serhiy.storchaka
Date 2013-12-02.17:05:40
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1386003940.48.0.863443757354.issue19858@psf.upfronthosting.co.za>
In-reply-to
Content
I afraid this is not a feature, but a bug. pickletools.optimize() can produce invalid pickled data when *PUT and MEMOIZE opcodes are used.

>>> import pickle, pickletools
>>> p = b'\x80\x04]\x94(\x8c\x04spamq\x01\x8c\x03ham\x94h\x02e.'
>>> pickle.loads(p)
['spam', 'ham', 'ham']
>>> pickletools.dis(p)
    0: \x80 PROTO      4
    2: ]    EMPTY_LIST
    3: \x94 MEMOIZE
    4: (    MARK
    5: \x8c     SHORT_BINUNICODE 'spam'
   11: q        BINPUT     1
   13: \x8c     SHORT_BINUNICODE 'ham'
   18: \x94     MEMOIZE
   19: h        BINGET     2
   21: e        APPENDS    (MARK at 4)
   22: .    STOP
highest protocol among opcodes = 4
>>> p2 = pickletools.optimize(p)
>>> p2
b'\x80\x04\x95\x13\x00\x00\x00\x00\x00\x00\x00]\x94(\x8c\x04spam\x8c\x03ham\x94h\x02e.'
>>> pickle.loads(p2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 2
>>> pickletools.dis(p2)
    0: \x80 PROTO      4
    2: \x95 FRAME      19
   11: ]    EMPTY_LIST
   12: \x94 MEMOIZE
   13: (    MARK
   14: \x8c     SHORT_BINUNICODE 'spam'
   20: \x8c     SHORT_BINUNICODE 'ham'
   25: \x94     MEMOIZE
   26: h        BINGET     2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/pickletools.py", line 2458, in dis
    raise ValueError(errormsg)
ValueError: memo key 2 has never been stored into
History
Date User Action Args
2013-12-02 17:05:40serhiy.storchakasetrecipients: + serhiy.storchaka, alexandre.vassalotti
2013-12-02 17:05:40serhiy.storchakasetmessageid: <1386003940.48.0.863443757354.issue19858@psf.upfronthosting.co.za>
2013-12-02 17:05:40serhiy.storchakalinkissue19858 messages
2013-12-02 17:05:40serhiy.storchakacreate