Author nirs
Recipients brett.cannon, nirs, serhiy.storchaka, vstinner, yselivanov
Date 2017-11-04.22:32:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1509834750.61.0.213398074469.issue31945@psf.upfronthosting.co.za>
In-reply-to
Content
blocksize is hardcoded to 8192 in send() and _read_readable(), preventing
efficient upload when using file-like body.

Users of the module that are not interested in chunked encoding can rewrite
the copy loop using HTTPConnection.send():

   conn = HTTPSConnection(...)
   conn.putrequest(...)
   conn.putheader(...)
   conn.endheaders()

   while True:
       chunk = file.read(512*1024)
       if not chunk:
          break
       conn.send(chunk)

But fixing send() to use a configurable blocksize seems more useful.

Also, users of requests do not have access the underlying connection, so
they cannot use preferred buffer size.

When reading from /dev/zero and uploading to server that drop the received
data, larger buffer size gives 3X more throughput *and* 1/3 of cpu time.
With real storage and network, the effect will probably be much smaller.
History
Date User Action Args
2017-11-04 22:32:30nirssetrecipients: + nirs, brett.cannon, vstinner, serhiy.storchaka, yselivanov
2017-11-04 22:32:30nirssetmessageid: <1509834750.61.0.213398074469.issue31945@psf.upfronthosting.co.za>
2017-11-04 22:32:30nirslinkissue31945 messages
2017-11-04 22:32:30nirscreate