Message188280
The latest framing patch looks pretty nice overall. One concern is we need to make sure the C implementation call _Pickler_OpcodeBoundary often enough to keep the frames around the sizes. For example, batch_save_list and batch_save_dict can currently create a frame much larger than expected. Interestingly enough, I found pickle, with patch applied, crashes when handling such frames:
13:44:43 pep-3154 $ ./python -c "import pickle, io; pickle.dump(list(range(10**5)), io.BytesIO(), 4)"
Debug memory block at address p=0x1e96b10: API 'o'
52 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x1e96b44 are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x00 *** OUCH
at tail+1: 0x00 *** OUCH
at tail+2: 0x00 *** OUCH
at tail+3: 0x00 *** OUCH
at tail+4: 0x4d *** OUCH
at tail+5: 0x75 *** OUCH
at tail+6: 0x5b *** OUCH
at tail+7: 0xfb
The block was made by call #237465 to debug malloc/realloc.
Data at p: 00 00 00 00 00 00 00 00 ... ff ff ff ff 00 00 00 00
Fatal Python error: bad trailing pad byte
Current thread 0x00007f5dea491700:
File "<string>", line 1 in <module>
Aborted (core dumped)
Also, I think we should try to make pickletools.dis display the frame boundaries to help with debugging. This could be implemented by adding an option to pickletools.genops which could be helpful for testing the framing implementation as well. |
|
Date |
User |
Action |
Args |
2013-05-02 21:00:01 | alexandre.vassalotti | set | recipients:
+ alexandre.vassalotti, rhettinger, pitrou, Arfrever, asvetlov, neologix, serhiy.storchaka |
2013-05-02 21:00:01 | alexandre.vassalotti | set | messageid: <1367528401.47.0.364612925247.issue17810@psf.upfronthosting.co.za> |
2013-05-02 21:00:01 | alexandre.vassalotti | link | issue17810 messages |
2013-05-02 21:00:01 | alexandre.vassalotti | create | |
|