classification
Title: IPv6 Interface naming/indexing functions
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: gregory.p.smith Nosy List: OG7, christian.heimes, gregory.p.smith, haypo, meiermic, nadeem.vawda, neologix, pitrou, python-dev, santoso.wijaya
Priority: normal Keywords: patch

Created on 2007-07-02 16:36 by meiermic, last changed 2011-05-20 15:27 by neologix. This issue is now closed.

Files
File name Uploaded Description Edit
iffunctions.diff meiermic, 2007-07-02 16:36 Patch for Library Code & Documentation review
socket_if.diff neologix, 2011-05-20 14:03 review
Messages (24)
msg52807 - (view) Author: Michael Meier (meiermic) Date: 2007-07-02 16:36
The Python socket module does not feature the functions declared in net/if.h even though they are essential to support IPv6 multicast to a reasonable extent. 

This patch adds a straightforward intertace to the three functions if_nameindex(), if_nametoindex() and if_indextoname().

I didn't add a testcase as I don't know what to test ;)


The patch seems to work on Ubuntu 7.04.
msg57772 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2007-11-23 08:21
How do you know that the patch is working when you don't know how to
test it? Nobody is going to apply new features without unit tests.
msg85686 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2009-04-07 06:40
What is the status of these methods?

On the windows side, it appears that these are only available in Vista
and later so we'll need some conditional compliation magic there:

 http://msdn.microsoft.com/en-us/library/bb408409(VS.85).aspx

....

fyi - the implementation can be simplified by using Py_BuildValue().
msg136011 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-15 07:27
New changeset c2515cb23d9b by Gregory P. Smith in branch 'default':
Issue #1746656: Add if_nameindex, if_nametoindex, if_indextoname
http://hg.python.org/cpython/rev/c2515cb23d9b
msg136012 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2011-05-15 07:29
I added these with conditional compilation via autoconf for use on posix systems.  These methods are not IPv6 specific.

Anyone who wants to see them supported on windows will need to add whatever conditional compilation magic is required to enable that on appropriate builds.
msg136016 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-15 07:56
> New changeset c2515cb23d9b by Gregory P. Smith in branch 'default':
> Issue #1746656: Add if_nameindex, if_nametoindex, if_indextoname

The _socket module doesn't compile on OpenIndiana anymore:
http://www.python.org/dev/buildbot/all/builders/AMD64%20OpenIndiana%203.x/builds/1207/steps/test/logs/stdio
msg136019 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-15 08:02
> The _socket module doesn't compile on OpenIndiana anymore

Same problem of FreeBSD 8.2:
http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%208.2%203.x/builds/291/steps/test/logs/stdio
msg136026 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2011-05-15 09:59
Also failing to compile on OS X:

http://www.python.org/dev/buildbot/all/builders/AMD64%20Leopard%203.x/builds/1385/steps/test/logs/stdio

The problem seems to be that <net/if.h> is not being included on these
non-Linux systems. Looking at Modules/socketmodule.h reveals that its
inclusion is conditional on HAVE_NETPACKET_PACKET_H:

    #ifdef HAVE_NETPACKET_PACKET_H
    # include <sys/ioctl.h>
    # include <net/if.h>
    # include <netpacket/packet.h>
    #endif

Changing it to something like this should solve the problem:

    #ifdef HAVE_NET_IF_H
    # include <net/if.h>
    #endif

    #ifdef HAVE_NETPACKET_PACKET_H
    # include <sys/ioctl.h>
    # include <netpacket/packet.h>
    #endif
msg136027 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-15 11:28
New changeset 434dfe42fde1 by Nadeem Vawda in branch 'default':
Fix _socket compilation failures on non-Linux buildbots (cf. issue #1746656).
http://hg.python.org/cpython/rev/434dfe42fde1
msg136028 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2011-05-15 12:26
The OpenIndiana bots are now green, but the BSD and OS X bots are still
failing. It seems that on those systems, <net/if.h> depends on <sys/types.h> and maybe some other headers, so the current configure
script detects it as "present but not usable".

I'm currently setting up a FreeBSD VM so I can poke around and figure out
what the missing dependencies are.
msg136041 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2011-05-15 17:46
thanks i'll take a look at OS X here.  obviously i did development and
testing on linux.
msg136042 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2011-05-15 18:19
OK, that's great. It'll be another couple of hours before I can do
anything from the FreeBSD side; I'm still waiting for the ISO to finish
downloading :/
msg136046 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-15 19:18
New changeset b6aafb20e5f5 by Gregory P. Smith in branch 'default':
issue #1746656: Fix for OS X. configure and #include changes so that the socket
http://hg.python.org/cpython/rev/b6aafb20e5f5
msg136068 - (view) Author: Nadeem Vawda (nadeem.vawda) * (Python committer) Date: 2011-05-16 05:54
All the buildbots are back to normal. It looks like FreeBSD was having
the same problem as OS X.
msg136174 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-17 19:50
Sorry to reopen, but do these methods really have to return *byte strings* for interface names?
In Python 3, we usually take the position that textually meaningful data should be str, not bytes (even filenames under POSIX). The usual way to do this is to use the "filesystem encoding" (os.fsdecode in pure Python, PyUnicode_DecodeFSDefault in C).

Also, the tests are really minimal. They should at least check the type and structure of values returned by these functions.
msg136263 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-05-18 21:51
Here's a patch:
- those functions now accept and return str, not bytes arrays
- some of them were not declared static, it's now fixed
- use PyErr_SetFromErrno when errno is set
- add tests (return type, nonexistent interface name/index and invalid argument)
- cleanup

Note that I'm not sure about my char * <-> PyObject conversions, encoding/unicode issues are really something I'm completely oblivious to.
msg136271 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-19 09:46
> Here's a patch:
> - those functions now accept and return str, not bytes arrays

You use UTF-8 encoding:

+                "Is", ni[i].if_index, ni[i].if_name);
+    if (!PyArg_ParseTuple(args, "s:if_nametoindex", &ifname))

You should also use the FS encoding with surrogateescape error handler:

 - parse arguments using "O&" format with PyUnicode_FSConverter: it gives you a PyBytes object, then use PyBytes_AS_STRING() and PyBytes_GET_SIZE()
 - use PyUnicode_DecodeFSDefault() to decode a byte string
msg136272 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-19 09:49
Example with a non-ASCII interface name:

$ sudo tunctl -u haypo -t unicodeé
Set 'unicodeé' persistent and owned by uid 1000

$ sudo ifconfig -a|grep unicode|hexdump -C
00000000  75 6e 69 63 6f 64 65 c3  a9 20 4c 69 6e 6b 20 65  |unicode.. Link e|
00000010  6e 63 61 70 3a 45 74 68  65 72 6e 65 74 20 20 48  |ncap:Ethernet  H|
00000020  57 61 64 64 72 20 64 36  3a 30 38 3a 31 63 3a 65  |Waddr d6:08:1c:e|
00000030  30 3a 33 33 3a 30 36 20  20 0a                    |0:33:06  .|
0000003a

So in my setup (UTF-8 locale encoding), U+00E9 is encoded as {0xc3, 0xa9} (UTF-8).
msg136374 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-05-20 14:03
> You use UTF-8 encoding:

Here's an updated patch taking your comments into account (I'm really blissfully ignorant when it comes to encoding issues, so I hope it will be OK this time).
msg136376 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2011-05-20 14:22
that patch looks good.  though I do wish we had a function similar to PyObject_AsStringEncodedFSDefault() so that the ParseTuple call wasn't needed for this relatively common operation when interfacing with system library calls that deal in C strings.
msg136377 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-05-20 14:38
> that patch looks good

I tested it with non-ASCII interface names: it works as expected with
ASCII and UTF-8 locales. It's the first time that I see the "O&" format
for Py_BuildValue(), I didn't know this one. The patch looks good.

@neologix: You can commit it into Python 3.3. Tell me if you need
help ;-)

> though I do wish we had a function similar to
> PyObject_AsStringEncodedFSDefault() so that the ParseTuple call wasn't
> needed for this relatively common operation when interfacing with
> system library calls that deal in C strings.

PyUnicode_FSConverter and PyUnicode_FSDecoder don't fit your needs?

What you would be the purpose of PyObject_AsStringEncodedFSDefault()?
What is its prototype?
msg136378 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-20 14:54
New changeset cc60d0283fad by Charles-François Natali in branch 'default':
Issue #1746656: make if_nameindex(), if_indextoname() and if_nametoindex()
http://hg.python.org/cpython/rev/cc60d0283fad
msg136384 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-05-20 15:18
> @neologix: You can commit it into Python 3.3. Tell me if you need
> help ;-)

My first commit :-)
What's the next step?
Can this issue be closed, or should I wait until the tests pass on
some buildbot?
msg136385 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-20 15:20
> My first commit :-)
> What's the next step?
> Can this issue be closed, or should I wait until the tests pass on
> some buildbot?

You can close the issue nevertheless (as "fixed" /
"committed/rejected"). It can be reopened later if there's a buildbot
failure.
(note that many 3.x buildbots are currently red for another reason :))
History
Date User Action Args
2011-05-20 15:27:38neologixsetstatus: open -> closed
2011-05-20 15:20:40pitrousetmessages: + msg136385
2011-05-20 15:18:51neologixsetmessages: + msg136384
2011-05-20 14:54:56python-devsetmessages: + msg136378
2011-05-20 14:38:49hayposetmessages: + msg136377
2011-05-20 14:22:07gregory.p.smithsetmessages: + msg136376
2011-05-20 14:03:29neologixsetfiles: + socket_if.diff

messages: + msg136374
2011-05-20 14:01:15neologixsetfiles: - socket_if.diff
2011-05-19 09:49:55hayposetmessages: + msg136272
2011-05-19 09:46:53hayposetmessages: + msg136271
2011-05-18 23:13:07santoso.wijayasetnosy: + santoso.wijaya
2011-05-18 21:51:31neologixsetfiles: + socket_if.diff
nosy: + neologix
messages: + msg136263

2011-05-17 19:50:42pitrousetstatus: closed -> open
nosy: + pitrou
messages: + msg136174

2011-05-16 05:54:23nadeem.vawdasetmessages: + msg136068
2011-05-15 19:18:37python-devsetmessages: + msg136046
2011-05-15 18:19:20nadeem.vawdasetmessages: + msg136042
2011-05-15 17:46:56gregory.p.smithsetmessages: + msg136041
2011-05-15 12:26:35nadeem.vawdasetmessages: + msg136028
2011-05-15 11:28:06python-devsetmessages: + msg136027
2011-05-15 09:59:26nadeem.vawdasetnosy: + nadeem.vawda
messages: + msg136026
2011-05-15 08:02:28hayposetmessages: + msg136019
2011-05-15 07:56:02hayposetnosy: + haypo
messages: + msg136016
2011-05-15 07:29:33gregory.p.smithsetstatus: open -> closed
versions: + Python 3.3, - Python 3.2
messages: + msg136012

resolution: fixed
stage: test needed -> resolved
2011-05-15 07:27:04python-devsetnosy: + python-dev
messages: + msg136011
2010-08-09 04:11:30terry.reedysetversions: + Python 3.2, - Python 3.1, Python 2.7
2009-04-07 06:40:20gregory.p.smithsetassignee: gregory.p.smith

messages: + msg85686
nosy: + gregory.p.smith
2009-04-07 04:05:26ajaksu2setstage: test needed
versions: + Python 3.1, Python 2.7, - Python 2.6
2007-11-23 08:21:39christian.heimessetnosy: + christian.heimes
messages: + msg57772
2007-10-07 16:29:34OG7setnosy: + OG7
type: enhancement
2007-07-02 16:36:34meiermiccreate