http.client.HTTPResponse._readinto_chunked has two problems in python 3.8 - 3.9.
1. _safe_readinto assumes that self.fp.readinto(b) will read exactly len(b) bytes. This is not always true, especially in case of SSL traffic. But _safe_readinto raises IncompleteRead if less len(b) bytes was read.
So, _safe_readinto should be removed and substituted with self.fp.readinto
2. _readinto_chunked may lose chunked block boundary because of this line:
self.chunk_left = 0
it should be changed to
self.chunk_left = chunk_left - n
in order to self._get_chunk_left() be able to find real chunk boundary
Corrected function looks like this:
def _readinto_chunked(self, b):
assert self.chunked != _UNKNOWN
total_bytes = 0
mvb = memoryview(b)
try:
while True:
chunk_left = self._get_chunk_left()
if chunk_left is None:
return total_bytes
if len(mvb) <= chunk_left:
n = self.fp.readinto(mvb)
self.chunk_left = chunk_left - n
return total_bytes + n
temp_mvb = mvb[:chunk_left]
n = self.fp.readinto(temp_mvb)
mvb = mvb[n:]
total_bytes += n
self.chunk_left = chunk_left - n
except IncompleteRead:
raise IncompleteRead(bytes(b[0:total_bytes]))
|