This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Use socket.sendall()/send() send data larger than 2GB will be truncated and return None, without exception raised.
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: kmiku7, matrixise, remi.lapeyre, vstinner
Priority: normal Keywords: patch

Created on 2019-03-18 07:12 by kmiku7, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12397 merged matrixise, 2019-03-18 09:11
Messages (10)
msg338166 - (view) Author: kmiku7 (kmiku7) Date: 2019-03-18 07:12
In file Modules/socketmodule.c, sock_send/sock_sendall use int to keep array length need be sent and has sent.
msg338170 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-03-18 08:01
Related issue : issue18100 . Seems this was fixed in 3.x .
msg338174 - (view) Author: kmiku7 (kmiku7) Date: 2019-03-18 08:29
Can we fix it in 2.7?
msg338182 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-03-18 09:13
2.7 is in bugfix mode, but I have created a PR (not yet completed, need to be tested with real unittest).

Could you try my PR and why not produce a script for the tests.

Thank you.
msg338186 - (view) Author: kmiku7 (kmiku7) Date: 2019-03-18 09:55
Thanks for your contribute! I just have read your PR on github.
Because I am not familiar about how to contribute a patch, I doubt it's reasonable to write a test which will consuming more than 2GB memory space? Because if write a test to cover this case, we will build a string larger than 2GB, and use socket to send it. Dose the machine running tests have enough memory resource?
msg338187 - (view) Author: Rémi Lapeyre (remi.lapeyre) * Date: 2019-03-18 10:02
Hi @kmiku7, it's possible to write tests that use a large amount of memory and to run them only when asked to with bigmemtest: https://docs.python.org/3/library/test.html#test.support.bigmemtest

There is some tests that use it in the codebase like https://github.com/python/cpython/blob/1561703a78849ac3511055590d9d1bd2c62a2072/Lib/test/test_bz2.py#L646-L647

Creating a large string and sending it through a socket should be ok if you mark the test with this decorator.
msg338225 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-18 14:03
@kmiku7: Did you find this issue in production or while testing Python? Or did you spot the idea while reading Python source code?
msg338226 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-18 14:05
> Because I am not familiar about how to contribute a patch, I doubt it's reasonable to write a test which will consuming more than 2GB memory space? Because if write a test to cover this case, we will build a string larger than 2GB, and use socket to send it. Dose the machine running tests have enough memory resource?

Most computers have at least 4 GB of memory. It would be possible to write a "bigmem" test which build a string longer than 2 GiB, send it and check the received length, but I'm not sure that it's worth it: there is no such test in the master branch.
msg338307 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 01:56
New changeset f70b884ad70e2ce762842ae469f88bd48fe13998 by Victor Stinner (Stéphane Wirtel) in branch '2.7':
bpo-36337: socket.send()/sendall() use Py_ssize_t (GH-12397)
https://github.com/python/cpython/commit/f70b884ad70e2ce762842ae469f88bd48fe13998
msg338308 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 01:57
Thanks Stéphane for the quick fix, I merged your PR. Thanks kmiku7 for your bug report.

Until the next Python 2.7 version is released, you can use Python 3 which is already fixed ;-)
History
Date User Action Args
2022-04-11 14:59:12adminsetgithub: 80518
2019-03-19 01:57:34vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg338308

stage: patch review -> resolved
2019-03-19 01:56:31vstinnersetmessages: + msg338307
2019-03-18 14:05:21vstinnersetmessages: + msg338226
2019-03-18 14:03:41vstinnersetnosy: + vstinner
messages: + msg338225
2019-03-18 10:02:33remi.lapeyresetnosy: + remi.lapeyre
messages: + msg338187
2019-03-18 09:55:48kmiku7setnosy: - vstinner, giampaolo.rodola, xtreak
messages: + msg338186
2019-03-18 09:13:16matrixisesetnosy: + matrixise
messages: + msg338182
2019-03-18 09:11:56matrixisesetkeywords: + patch
stage: patch review
pull_requests: + pull_request12352
2019-03-18 08:36:38xtreaksetnosy: + vstinner
2019-03-18 08:35:40xtreaksetnosy: + giampaolo.rodola
2019-03-18 08:29:45kmiku7setmessages: + msg338174
2019-03-18 08:01:42xtreaksetnosy: + xtreak
messages: + msg338170
2019-03-18 07:12:24kmiku7create