Author Arden
Recipients Arden
Date 2020-01-17.16:21:16
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1579278077.15.0.801446785359.issue39371@roundup.psfhosted.org>
In-reply-to
Content
http.client.HTTPResponse._readinto_chunked has two problems in python 3.8 - 3.9.

1. _safe_readinto assumes that self.fp.readinto(b) will read exactly len(b) bytes. This is not always true, especially in case of SSL traffic. But _safe_readinto raises IncompleteRead if less len(b) bytes was read.

So, _safe_readinto should be removed and substituted with self.fp.readinto

2. _readinto_chunked may lose chunked block boundary because of this line:
  self.chunk_left = 0
it should be changed to
  self.chunk_left = chunk_left - n
in order to self._get_chunk_left() be able to find real chunk boundary

Corrected function looks like this:
def _readinto_chunked(self, b):
    assert self.chunked != _UNKNOWN
    total_bytes = 0
    mvb = memoryview(b)
    try:
        while True:
            chunk_left = self._get_chunk_left()
            if chunk_left is None:
                return total_bytes

            if len(mvb) <= chunk_left:
                n = self.fp.readinto(mvb)
                self.chunk_left = chunk_left - n
                return total_bytes + n

            temp_mvb = mvb[:chunk_left]
            n = self.fp.readinto(temp_mvb)
            mvb = mvb[n:]
            total_bytes += n
            self.chunk_left = chunk_left - n

    except IncompleteRead:
        raise IncompleteRead(bytes(b[0:total_bytes]))
History
Date User Action Args
2020-01-17 16:21:17Ardensetrecipients: + Arden
2020-01-17 16:21:17Ardensetmessageid: <1579278077.15.0.801446785359.issue39371@roundup.psfhosted.org>
2020-01-17 16:21:17Ardenlinkissue39371 messages
2020-01-17 16:21:16Ardencreate