This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author steve.dower
Recipients paul.moore, steve.dower, tim.golden, zach.ware
Date 2020-09-17.20:25:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1600374337.91.0.20537040247.issue41806@roundup.psfhosted.org>
In-reply-to
Content
When a socket object has a non-zero timeout, all send/recv/etc. calls are preceded by a select() call before making the actual call. If there is no timeout, the select call is skipped.

It appears that on Windows, select() will block until the previous operation has completed, even if the next one would reliably succeed. For example, http.client uses a default blocksize of 8192 bytes (which is _really_ small for 2020, but probably not for 2002 when it was set - can we just increase it?), which will break up a stream into many little chunks.

With no timeout set: send() is called repeatedly and Windows buffers blocks internally, resulting in the actual sends being bigger (as seen with a network trace - they'd range from 8KB-50KB).

With a timeout set: select() is called and waits for the previous send() to complete, then calls the next send(). This results in every single send being exactly 8KB.

This latter case also results in every operation becoming essentially synchronous, instead of picking up some implied parallelism in the OS and network driver. (Yes, increasing the block size in the above example also helps, but doesn't prevent the issue in the first place, nor does it help with other cases.)

Has anyone ever looked into alternate ways to have the timeout work here? Such as SO_SNDTIMEO? I don't want to force through a full redesign here, but getting the select() call out of the main path here seems like an easy win.

Would anyone like to take a look? I'm happy to help mentor someone through a patch (as long as you're familiar with networking/sockets already).
History
Date User Action Args
2020-09-17 20:25:37steve.dowersetrecipients: + steve.dower, paul.moore, tim.golden, zach.ware
2020-09-17 20:25:37steve.dowersetmessageid: <1600374337.91.0.20537040247.issue41806@roundup.psfhosted.org>
2020-09-17 20:25:37steve.dowerlinkissue41806 messages
2020-09-17 20:25:37steve.dowercreate