Message241399
After cleaning my build and rebuilding with “./configure --with-pymalloc --with-pydebug”, I reduced my script to these four lines:
b1 = bytearray(b"abcdefghij") # 10 bytes
del b1[:1]
del b1[:1]
b1 += b"klmnopq" # 7 bytes
Patch bytearray-fix.patch fixes the bug by taking account fact that ob_start is offset into the allocated memory buffer when checking if a reallocation is necessary.
Explanation with the unpatched code and my four-line script above:
1. First line allocates 11 bytes of memory (10 for the byte string + 1 NUL terminator)
2. First “del” reduces the bytearray length to 9 bytes, but actually reallocates an expanded memory buffer of 16 bytes! (quirky but not the bug)
3. Second “del” reduces the bytearray length to 8 bytes, and increments an internal ob_start offset without any memory copying or reallocation. (Fine.)
4. Appending step needs to add 7 bytes to the 8-byte array. 7 + 8 + 1 is 16 bytes total required for bytearray and NUL terminator, but since ob_start is offset from the start of the allocated memory buffer, we overwrite past the end of the buffer.
Memory debugging output and extra debug printfs of my own:
Resizing to size 10 (current log. offset 0 alloc 0)
=> Major upsize, new alloc = 11
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 9 (current log. offset 1 alloc 11)
=> Moderate upsize, new alloc = 16
- Done assigning to linear slice
Assigning to linear slice
- Shifting ob_start for negative growth -1
Resizing to size 8 (current log. offset 1 alloc 16)
=> Minor downsize
- Done assigning to linear slice
Debug memory block at address p=0x7f1af630a0d0: API 'o'
16 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x7f1af630a0e0 are not all FORBIDDENBYTE (0xfb):
at tail+0: 0x00 *** OUCH
at tail+1: 0xfb
at tail+2: 0xfb
at tail+3: 0xfb
at tail+4: 0xfb
at tail+5: 0xfb
at tail+6: 0xfb
at tail+7: 0xfb
The block was made by call #32897 to debug malloc/realloc.
Data at p: 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71
Fatal Python error: bad trailing pad byte |
|
Date |
User |
Action |
Args |
2015-04-18 05:48:47 | martin.panter | set | recipients:
+ martin.panter, terry.reedy, pitrou, tim.golden, ned.deily, zach.ware, serhiy.storchaka, steve.dower, wolma, alexei.romanov, johan |
2015-04-18 05:48:47 | martin.panter | set | messageid: <1429336127.66.0.9266281443.issue23985@psf.upfronthosting.co.za> |
2015-04-18 05:48:47 | martin.panter | link | issue23985 messages |
2015-04-18 05:48:46 | martin.panter | create | |
|