classification
Title: Add method send_io, recv_io to the socket module.
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, inada.naoki, mdk, nanjekyejoannah, tokibito, vstinner
Priority: normal Keywords: patch

Created on 2016-11-17 15:05 by tokibito, last changed 2019-09-11 17:18 by vstinner.

Files
File name Uploaded Description Edit
socket_send_io_recv_io.patch tokibito, 2016-11-17 15:05 review
Pull Requests
URL Status Linked Edit
PR 12889 merged nanjekyejoannah, 2019-04-20 22:46
Messages (10)
msg281041 - (view) Author: Shinya Okano (tokibito) * Date: 2016-11-17 15:05
This patch makes it easy to pass file descriptor in using AF_UNIX.

Ruby language libraries have such methods.

see slso:
 - https://docs.ruby-lang.org/en/2.3.0/UNIXSocket.html#method-i-send_io
msg281065 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-11-17 23:02
Hi, thanks for your contribution!

Documentation give examples implementation of your methods:

 - https://docs.python.org/3/library/socket.html#socket.socket.sendmsg 
 - https://docs.python.org/3/library/socket.html#socket.socket.recvmsg

and from here, some remarks:

 - Why allowing people to mix fds and file objects? I'm not a fan of this: I prefer the clarity of allowing only file descriptors. Have you seen such methods (allowing both fd and file obj) in the stdlib?
 - The documented implementation for recv_io checks for integer truncation, there may be a good reason, should'n you do the same?
 - The documented implementation allows to pass a message, shouldn't you at least allow for an optional message to be passed?

Adding those methods may make sense if people are copying send_fds/recv_fds in their code, but I found only two copy/paste in github, so I think the best is to put your code as a module on pypi and see, from here, if it gets popular?

You should also write a few tests, and comply to the PEP8 (your two methods should probably be separated by a newline).

Bests
msg281089 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2016-11-18 10:08
FYI, multiprocessing.reduction module has send_fds and recv_fds.
msg282348 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-12-04 15:21
Unless somebody don't think so, I think this should go as a pypi module before going to the socket module, so this issue should probably be closed.
msg282415 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-12-05 12:41
Good idea! Initially I planned to add the functions and some other helpers around AF_UNIX to Python 3.6. You can directly copy the example functions from the documentation.

https://docs.python.org/3/library/socket.html#socket.socket.sendmsg
https://docs.python.org/3/library/socket.html#socket.socket.recvmsg

Please keep the names send_fds() and recv_fds(), and pass the message like in the examples. Some protocols use the payload to signal the count of fds. Your patch is also lacking unit tests and documentation update.
msg282488 - (view) Author: Shinya Okano (tokibito) * Date: 2016-12-06 00:09
Thanks for reviews and comments. I will work on the weekend.
msg282508 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-12-06 09:25
Take your time, feature freeze for 3.7 is in about 18 months from now.
msg346379 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-24 11:29
See also bpo-37385: "test_multiprocessing fails on AMD64 FreeBSD CURRENT Shared 2.7".
msg351986 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-11 17:12
New changeset 8d120f75fb8c8731464b5f7531d74cdbb897d924 by Victor Stinner (Joannah Nanjekye) in branch 'master':
bpo-28724: Add methods send_fds and recv_fds to the socket module (GH-12889)
https://github.com/python/cpython/commit/8d120f75fb8c8731464b5f7531d74cdbb897d924
msg351988 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-09-11 17:18
Thanks Shinya Okano for the original patch.

Well done Joannah! Thanks for your tenacity :-) This PR has 100 comments and 27 commits which shows the complexity of the feature.

Honestly, I'm not 100% happy with current documentation, but I chose to merge the PR anyway. Please leave this issue open until the documentation is completed to mention corner cases like partial send (similar to sock.send vs sock.sendall).
History
Date User Action Args
2019-09-11 17:18:17vstinnersetmessages: + msg351988
2019-09-11 17:12:24vstinnersetmessages: + msg351986
2019-08-28 23:46:16vstinnerlinkissue37385 superseder
2019-06-24 11:29:27vstinnersetnosy: + vstinner
messages: + msg346379
2019-04-20 22:49:26nanjekyejoannahsetnosy: + nanjekyejoannah
2019-04-20 22:46:46nanjekyejoannahsetstage: needs patch -> patch review
pull_requests: + pull_request12815
2016-12-06 09:25:36christian.heimessetmessages: + msg282508
2016-12-06 00:09:25tokibitosetmessages: + msg282488
2016-12-05 12:41:30christian.heimessetnosy: + christian.heimes

messages: + msg282415
stage: needs patch
2016-12-04 15:21:03mdksetmessages: + msg282348
2016-11-18 10:08:07inada.naokisetnosy: + inada.naoki
messages: + msg281089
2016-11-17 23:02:20mdksetnosy: + mdk
messages: + msg281065
2016-11-17 15:05:33tokibitocreate