This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author zveinn
Recipients zveinn
Date 2021-02-26.20:17:59
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1614370680.1.0.103129903648.issue43332@roundup.psfhosted.org>
In-reply-to
Content
Hey, some time ago I ran into some code in the cpython code I thought might be possible to improve it a little bit.

https://github.com/python/cpython/blob/master/Lib/http/client.py#L903

This code specifically. 

Notice how the self.send() method is used multiple time to construct the CONNECT request. When the network load is high, these different parts actually get split into separate network frames.

This causes additional meta data to be sent, effectively costing more bandwidth and causing the request to be split into multiple network frames.

This has some interesting behavior on the receiving end as well. 

If you send everything as a single network frame, then the receiver can read the entire thing in a single read call. If you send multiple frames, the main reader pipe now needs a temporary buffer to encapsulate the multiple calls. 

Because of this, sending requests as many network frames actually causes a rise in processing complexity on the receiving end. 

Here is a github issue I made about this problem some time ago: https://github.com/psf/requests/issues/5384
In this issue you will find detailed information and screenshots.

My recommendation would be to construct the query as a whole before using a single self.send() to send the whole payload in one network frame. Even if we ignore the added complexity on the receivers end, the gain in network performance is worth it.
History
Date User Action Args
2021-02-26 20:18:00zveinnsetrecipients: + zveinn
2021-02-26 20:18:00zveinnsetmessageid: <1614370680.1.0.103129903648.issue43332@roundup.psfhosted.org>
2021-02-26 20:18:00zveinnlinkissue43332 messages
2021-02-26 20:17:59zveinncreate