classification
Title: fix output string length for binascii.b2a_uu()
Type: crash Stage:
Components: Extension Modules Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: haypo, pitrou
Priority: normal Keywords: patch

Created on 2010-01-14 11:56 by haypo, last changed 2010-01-15 00:31 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
binascii_b2a_uu_length-3.patch haypo, 2010-01-14 17:25
Messages (6)
msg97759 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-01-14 11:56
binascii_b2a_uu() estimate the output string length using 2+bin_len*2.
It's almost correct... except for bin_len=1. The result is a memory
write into unallocated memory:

   $ ./python -c "import binascii; binascii.b2a_uu('x')"
   Debug memory block at address p=0x87da568: API 'o'
       33 bytes originally requested
       The 3 pad bytes at p-3 are FORBIDDENBYTE, as expected.
       The 4 pad bytes at tail=0x87da589 are not all FORBIDDENBYTE (0xfb):
           at tail+0: 0x0a *** OUCH
           at tail+1: 0xfb
           at tail+2: 0xfb
           at tail+3: 0xfb
       The block was made by call #25195 to debug malloc/realloc.
       Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 21 3e 20 20 20
   Fatal Python error: bad trailing pad byte
   Abandon

Current output string length estimation for input string 0..10:

    >>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+bin_len*2) for bin_len in xrange(10)]
    [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

The estimation is correct for all lengths... except for bin_len=1. And
it's oversized for bin_len >= 9. The exact length is:

    2+ceil(bin_len*8/6) <=> 2+(bin_len+5)*8//6 <=> 2+(bin_len+2)*4//3

Example with length 0..10:

    >>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+(bin_len+2)*4//3) for bin_len in xrange(10)]
    [4, 6, 7, 8, 10, 11, 12, 14, 15, 16]

Attached patch uses the correct estimation.
msg97760 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-14 12:32
>>> [len(binascii.b2a_uu("x"*bin_len)) for bin_len in xrange(10)]
    [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
    >>> [(2+(bin_len+2)*4//3) for bin_len in xrange(10)]
    [4, 6, 7, 8, 10, 11, 12, 14, 15, 16]

How is this the correct estimation? The results are different.

Try the following:

>>> [(2+(bin_len+2)//3*4) for bin_len in xrange(10)]
[2, 6, 6, 6, 10, 10, 10, 14, 14, 14]
msg97764 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-01-14 13:43
> How is this the correct estimation? The results are different.

The estimation have be bigger or equal, but not smaller.

> Try the following:
> >>> [(2+(bin_len+2)//3*4) for bin_len in xrange(10)]
> [2, 6, 6, 6, 10, 10, 10, 14, 14, 14]

Cool, it's not an estimation but the exact result :-) I prefer to leave the resize unchanged. The new patch uses your "estimation" ;-)
msg97775 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-14 17:09
The patch doesn't apply cleanly against trunk (due to today's commits I fear, sorry).
Also, it would be nice to add a test.
msg97777 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-01-14 17:25
> The patch doesn't apply cleanly against trunk

Because of r77497 (issue #770). No problem, here is the new patch. I'm now using a git-svn repository to keep all my patches. It's much easier to update them to trunk ;-)

> Also, it would be nice to add a test.

done
msg97796 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-01-15 00:31
Patch committed in r77506, r77507, r77508 and r77509. Thank you!
History
Date User Action Args
2010-01-15 00:31:38pitrousetstatus: open -> closed
resolution: fixed
messages: + msg97796
2010-01-14 17:25:49hayposetfiles: - binascii_b2a_uu_length-2.patch
2010-01-14 17:25:44hayposetfiles: - binascii_b2a_uu_length.patch
2010-01-14 17:25:39hayposetfiles: + binascii_b2a_uu_length-3.patch

messages: + msg97777
2010-01-14 17:09:35pitrousetmessages: + msg97775
2010-01-14 13:43:38hayposetfiles: + binascii_b2a_uu_length-2.patch

messages: + msg97764
2010-01-14 12:32:55pitrousetnosy: + pitrou
messages: + msg97760
2010-01-14 11:56:21haypocreate