New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use sendfile where possible in httplib #57768
Comments
HTTPConnection.send() should use os.sendfile when possible to avoid copying data into userspace and back. |
This is not possible for two reasons:
I think we can use sendfile() in ftplib.py though . |
2011/12/8 Giampaolo Rodola' <report@bugs.python.org>:
I presume you could check for a binary mode, though? Also, you can |
ftplib's sendfile support is not tracked as bpo-13559. |
Ops! I meant bpo-13564. |
Patch in attachment uses the newly added socket.sendfile() method (bpo-17552). |
The multiple personalities of HTTPConnection.send() and friends is a bit of a can of worms. I suggest working on bpo-23740 to get an idea of what kinds of file objects are meant to be supported, and what things may work by accident and be used in the real world. For instance, is it possible to manually set Content-Length, and say supply a GzipFile reader, or file object positioned halfway through the file? How does this interact with the socket.sendfile() call? |
This issue is no newcomer friendly, I remove the "easy" keyword. |
I would like to take a stab at this. Giampaolo, would it be okay if I made a pull request updated from your patch? With the appropriate "Co-authored-by: Author Name <email_address>" line. |
Alex, https://bugs.python.org/issue23740 is identified as a dependency on this issue. We will have to resolve that first, and come back to this. And yes, if you contribute on other's patch, both the contributions will be included and appropriately credited. |
To check my understanding Is the motivation for the closer to
My guess is 5. |
Yes, the point number 5. We will have to evaluate if sendfile side-steps and avoids the issues noted in bpo-23740 |
sendfile() only works for plain HTTP. For technical reasons it does not work for HTTPS (*). These days majority of services use HTTPS. Therefore the usefulness of sendfile() patch is minimal. (*) It is possible to use sendfile() for TLS connections, but the feature requires a Kernel module that provides kTLS offloading feature, https://www.kernel.org/doc/html/latest/networking/tls-offload.html . In user space it requires OpenSSL 3.0.0 with kTLS support. 3.0.0 is currently under development. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: