classification
Title: ipaddress.IPv6Network.hosts function omits network and broadcast addresses
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: pmoody Nosy List: m01, ncoghlan, pmoody, python-dev
Priority: normal Keywords: patch

Created on 2013-10-03 22:51 by m01, last changed 2014-03-11 22:09 by ncoghlan. This issue is now closed.

Files
File name Uploaded Description Edit
ipaddress_ipv6_hosts.diff m01, 2013-10-03 22:51 naive implementation of proposed fix #1 review
Messages (10)
msg198927 - (view) Author: Michiel (m01) * Date: 2013-10-03 22:51
(See also: http://stackoverflow.com/q/19159168/1298153. This is my first bug submission)
Contrary to IPv4, IPv6 does not have a concept of network and broadcast addresses. It looks like the same code is used to generate the hosts for both IPv4 and IPv6, resulting in the network and broadcast addresses being omitted in IPv6, even though these are valid IPv6 addresses, albeit they may have a special meaning.

Repro:

$ python3
Python 3.3.2 (default, Sep  6 2013, 09:30:10) 
[GCC 4.8.1 20130725 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ipaddress
>>> print("\n".join([str(x) for x in ipaddress.ip_network("2001:0db8::/120").hosts()]))
2001:db8::1
2001:db8::2
...
2001:db8::fe
>>> 
>>> hex(int(ipaddress.ip_address('2001:db8::fe')))
'0x20010db80000000000000000000000fe'

The v4 docs state, that the hosts() function...
"Returns an iterator over the usable hosts in the network. The usable hosts are all the IP addresses that belong to the network, except the network address itself and the network broadcast address."

Assuming hosts excludes routers, this should probably return a generator/an iterator covering all addresses in the network, apart from the very first one (all 0's after the prefix), since this is typically the subnet-router anycast address (http://tools.ietf.org/html/rfc4291#section-2.6.1).

While the top range (including the currently omitted prefix + "all 1's" address) contains reserved anycast addresses (http://tools.ietf.org/html/rfc2526), I'm not sure whether excluding them is a great idea, as in the future some of these may become valid (anycast) host addresses.

Backwards compatibility considerations:
The proposed fix would add one extra address to the ones already returned. I think this is less likely to break code, than, say, removing all the reserved anycast addresses.

Implementation options:
1. override _BaseNetwork hosts() implementation in IPv6Network
2. add an is_host_address(ip_address) method, or something along those lines, to IPv4 and IPv6Address classes, and change the hosts() function to only return addresses for which is_host_address is True. This function might be generally useful, but the implementation is likely to be slower than 1.
3. use address_exclude perhaps..

I'm sure there are many other ways to implement the fix, but I've attached a "naive" diff. I'm not sure if there's a better way to implement it that doesn't involve copying code from hosts()/__iter__ and slightly tweaking it.

With 1:
>>> print("\n".join([str(x) for x in ipaddress.ip_network("2001:0db8::/124").hosts()]))
2001:db8::1
2001:db8::2
2001:db8::3
2001:db8::4
2001:db8::5
2001:db8::6
2001:db8::7
2001:db8::8
2001:db8::9
2001:db8::a
2001:db8::b
2001:db8::c
2001:db8::d
2001:db8::e
2001:db8::f
msg209927 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-02-01 23:45
Peter, could you take a look at this one? The status quo seems reasonable to me (and I assume to you since the stdlib ipaddress matches the way ipaddr handles this case), but there are details to Michiel's proposal that I'm not able to adequately assess.

However, also changing the target version to 3.5 - even if this behaviour was tweaked, it's unlikely to be something we would adjust in a maintenance release.
msg209929 - (view) Author: pmoody (pmoody) * (Python committer) Date: 2014-02-02 01:12
Ack. My first impression is that #1 is probably the right way to do this. I'm arguing with hg about the right way to stash a change, but I'll get this fixed.
msg210667 - (view) Author: pmoody (pmoody) * (Python committer) Date: 2014-02-08 17:10
Since there's no broadcast address for ipv6, should calling .broadcast* on an ipv6 address raise an exception?
msg210668 - (view) Author: pmoody (pmoody) * (Python committer) Date: 2014-02-08 17:25
Hey Michiel, the patch looks good to me. Have you signed the contributor license agreement?

http://www.python.org/psf/contrib/contrib-form/

Nick, is there anyway for me to check if this has been signed?
msg210836 - (view) Author: Michiel (m01) * Date: 2014-02-10 14:19
Hey Peter,

Cool, thanks for the feedback!
Looks like my email replies didn't get attached to the bug tracker..

I've just signed the form.

Regarding the .broadcast* question, I think that's probably best handled in a separate issue, but I think there are two other options in addition to the one you suggested: a) ensuring the .broadcast_address() function isn't actually defined for IPv6Networks (this would also trigger an exception if someone tried to call it, like you suggested) or b) making it return the link-local all-nodes multicast address. Personally speaking I'd probably favour option a, but I'm still new to Python standard library development ;-)
msg213150 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-11 16:57
New changeset b6271cbcc762 by Peter Moody in branch 'default':
Issue #19157: Include the broadcast address in the usuable hosts for IPv6
http://hg.python.org/cpython/rev/b6271cbcc762
msg213175 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-11 21:26
Peter, just confirming: you consider this a bug fix rather than a new
feature? I ask as default is currently still 3.4.1, which I find slightly
confusing myself (it will switch to 3.5 after the 3.4.0 final release this
weekend)
msg213177 - (view) Author: pmoody (pmoody) * (Python committer) Date: 2014-03-11 21:54
Yes, I think omitting the broadcast address in the usable hosts was a bug.
msg213179 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-03-11 22:09
Cool, thanks for clarifying.
History
Date User Action Args
2014-03-11 22:09:43ncoghlansetmessages: + msg213179
2014-03-11 21:54:12pmoodysetmessages: + msg213177
2014-03-11 21:26:00ncoghlansetmessages: + msg213175
2014-03-11 16:57:50pmoodysetstatus: open -> closed
2014-03-11 16:57:21python-devsetnosy: + python-dev
messages: + msg213150
2014-02-10 14:19:43m01setmessages: + msg210836
2014-02-08 17:25:36pmoodysetmessages: + msg210668
2014-02-08 17:10:21pmoodysetmessages: + msg210667
2014-02-02 01:12:36pmoodysetassignee: pmoody
messages: + msg209929
2014-02-01 23:45:53ncoghlansetnosy: + pmoody

messages: + msg209927
versions: + Python 3.5, - Python 3.3, Python 3.4
2014-01-31 22:37:12yselivanovsetnosy: + ncoghlan
2013-10-03 22:51:31m01create