Title: socket.fromfd()'s API is difficult or impossible to use correctly in general
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: aragilar, christian.heimes, cks, glyph, jcea, nascheme
Priority: normal Keywords:

Created on 2013-07-06 23:33 by cks, last changed 2016-06-24 17:22 by nascheme.

Messages (8)
msg192500 - (view) Author: Chris Siebenmann (cks) Date: 2013-07-06 23:33
socket.fromfd() requires you to supply at least the family and type of
the file descriptor that you are turning into a Python socket object.
However the socket module provides no documented way to determine what
these actually are and there are any number of situations where you may
get handed file descriptors with an indeterminate type. Nor does
providing incorrect family and type values raise an exception from
fromfd(), although if you get the wrong values you may get exceptions
from calling methods on the returned socket (you can also get garbled
and nonsensical results from some methods).

(For instance, in Python 3 calling .getsockname() may raise
UnicodeDecodeError under some circumstances.)

Suggested resolution: socket.fromfd() should always determine the
family and type itself and the arguments for them should become
optional. If they are supplied and they do not match the determined
family and type, fromfd() should probably raise an exception
(although that will break some existing code).

Less appealing resolution: the socket module should provide officially
documented and supported methods for determining the family and type
of a (socket) file descriptor. I think that a new module function to
do this would be the cleanest approach.
msg192501 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-07-07 00:01
Unfortunately it's not as easy as you may think. BSD sockets have no portable API to retrieve domain, type and protocol from a file descriptor. getsockopt() may be able to retrieve some or even all values but it's not portable. For example SO_DOMAIN and SO_PROTOCOL requires Linux 2.6.32 or newer. I'm not sure about BSD or Windows.

It's also not possible to verify the parameters until you actually do an operation like send(), recv() or accept() on a fd. Wrong parameters may not raise an error after all.

For now I suggest that you pass all information to the other process, that is fd, domain, type, proto. They are just integers.
msg192510 - (view) Author: Chris Siebenmann (cks) Date: 2013-07-07 03:04
As far as I know, you can recover everything except the protocol
portably on Unix (and fromfd() will already handwave the protocol).
getsockopt() with SO_TYPE will give you the type. The family can be
recovered by calling getsockname() with a plain struct sockaddr and
then examining sockaddr.sa_family.

As for the other suggestion:

When Python is plugging into larger systems or APIs that pass sockets
around, it is not possible to modify those APIs to pass the additional
information that Python wants (and other environments don't need).
As far as I know, no current protocol that passes file descriptors to
newly started programs or passes file descriptors over Unix sockets
with sendmsg() and SCM_RIGHTS does this. The people responsible for
these protocols are not likely to change them to accommodate Python's
wants. A Python program (and thus Python programmers) get to deal with
the issue themselves if they want Python to participate in these
systems and protocols.

I do want to be able to write Python programs that can interact in
these environments. It is possible to deal with this in C; it is
even possible to hack around this in Python today. I believe that
it should be officially supported.
msg192667 - (view) Author: Glyph Lefkowitz (glyph) (Python triager) Date: 2013-07-08 16:40
It would be nice for this to be fixed in a 2.7.x release too, if possible, since the workaround involves a ton of extra system calls and there's no other way to construct a socket object directly.
msg203183 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-11-17 14:43
Do you want to work on a patch for 3.4? You have about five days until 3.4 is going into feature freeze.
msg264650 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-05-02 14:31
fromfd() can be simplified after #26907 has landed.
msg269198 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2016-06-24 17:20
I've created a patch to add fromfd2().  I think it addresses the original concern of this issue.  My patch also adds the constants suggested by Issue #26907.
msg269199 - (view) Author: Neil Schemenauer (nascheme) * (Python committer) Date: 2016-06-24 17:22
Sorry, forgot to link the patch: Issue #27377.
Date User Action Args
2016-06-24 17:22:46naschemesetmessages: + msg269199
2016-06-24 17:20:33naschemesetnosy: + nascheme
messages: + msg269198
2016-05-02 14:31:32christian.heimessetmessages: + msg264650
versions: + Python 3.6, - Python 2.7, Python 3.4
2015-05-19 06:43:26aragilarsetnosy: + aragilar
2013-11-20 15:35:44jceasetnosy: + jcea
2013-11-17 14:43:17christian.heimessetmessages: + msg203183
2013-07-08 16:40:39glyphsetnosy: + glyph

messages: + msg192667
versions: + Python 2.7
2013-07-07 03:04:18ckssetmessages: + msg192510
2013-07-07 00:01:01christian.heimessetversions: + Python 3.4
nosy: + christian.heimes

messages: + msg192501

type: behavior -> enhancement
2013-07-06 23:33:52ckscreate