Author nikratio
Recipients benjamin.peterson, hynek, loewis, nikratio, pitrou, python-dev, stutzbach, vstinner
Date 2014-06-08.22:14:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1402265665.77.0.666050533041.issue20578@psf.upfronthosting.co.za>
In-reply-to
Content
Thanks for taking the time, and apologies about the test failure. I was probably too eager and ran only the test_io suite instead of everything.

I looked at the failure, and the problem is that the default Python BufferedIOBase.readinto implementation is semi-broken. It should work with any object implementing the memoryview protocol (like the C implementation), but it really only works with bytearray objects. The testIteration test only worked (prior to the patch) because there is a special case for array objects with format 'b':

        try:
            b[:n] = data
        except TypeError as err:
            import array
            if not isinstance(b, array.array):
                raise err
            b[:n] = array.array('b', data)

In other words, trying to read into any other object has always failed. In particular, even format code 'B' fails:

>>> import _pyio
>>> from array import array
>>> buf = array('b', b'x' * 10)
>>> _pyio.open('/dev/zero', 'rb').readinto(buf) 
10
>>> buf = array('B', b'x' * 10)
>>> _pyio.open('/dev/zero', 'rb').readinto(buf)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nikratio/clones/cpython/Lib/_pyio.py", line 1096, in readinto
    buf[:len_] = array.array('b', buf2)
TypeError: bad argument type for built-in operation


The readline implementation that my patch adds for BufferedReader does not contain this special case, and therefore with the patch even the test with a 'b'-array fails. 

For now, I've added the same special casing of 'b'-type arrays to the _readline() implementation in BufferedReader. This fixes the immediate problem (and this time I definitely ran the entire testsuite).

However, the fix is certainly not what I would consider a good solution.. but I guess that would better be addressed by a separate patch that also fixes the same issue in BufferedIOBase?
History
Date User Action Args
2014-06-08 22:14:25nikratiosetrecipients: + nikratio, loewis, pitrou, vstinner, benjamin.peterson, stutzbach, python-dev, hynek
2014-06-08 22:14:25nikratiosetmessageid: <1402265665.77.0.666050533041.issue20578@psf.upfronthosting.co.za>
2014-06-08 22:14:25nikratiolinkissue20578 messages
2014-06-08 22:14:25nikratiocreate