This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author alexandre.vassalotti
Recipients Arfrever, alexandre.vassalotti, asvetlov, neologix, pitrou, rhettinger, serhiy.storchaka
Date 2013-05-02.21:00:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1367528401.47.0.364612925247.issue17810@psf.upfronthosting.co.za>
In-reply-to
Content
The latest framing patch looks pretty nice overall. One concern is we need to make sure the C implementation call _Pickler_OpcodeBoundary often enough to keep the frames around the sizes. For example, batch_save_list and batch_save_dict can currently create a frame much larger than expected. Interestingly enough, I found pickle, with patch applied, crashes when handling such frames:

13:44:43 pep-3154 $ ./python -c "import pickle, io; pickle.dump(list(range(10**5)), io.BytesIO(), 4)"
Debug memory block at address p=0x1e96b10: API 'o'
    52 bytes originally requested
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x1e96b44 are not all FORBIDDENBYTE (0xfb):
        at tail+0: 0x00 *** OUCH
        at tail+1: 0x00 *** OUCH
        at tail+2: 0x00 *** OUCH
        at tail+3: 0x00 *** OUCH
        at tail+4: 0x4d *** OUCH
        at tail+5: 0x75 *** OUCH
        at tail+6: 0x5b *** OUCH
        at tail+7: 0xfb
    The block was made by call #237465 to debug malloc/realloc.
    Data at p: 00 00 00 00 00 00 00 00 ... ff ff ff ff 00 00 00 00
Fatal Python error: bad trailing pad byte

Current thread 0x00007f5dea491700:
  File "<string>", line 1 in <module>
Aborted (core dumped)

Also, I think we should try to make pickletools.dis display the frame boundaries to help with debugging. This could be implemented by adding an option to pickletools.genops which could be helpful for testing the framing implementation as well.
History
Date User Action Args
2013-05-02 21:00:01alexandre.vassalottisetrecipients: + alexandre.vassalotti, rhettinger, pitrou, Arfrever, asvetlov, neologix, serhiy.storchaka
2013-05-02 21:00:01alexandre.vassalottisetmessageid: <1367528401.47.0.364612925247.issue17810@psf.upfronthosting.co.za>
2013-05-02 21:00:01alexandre.vassalottilinkissue17810 messages
2013-05-02 21:00:01alexandre.vassalotticreate