classification
Title: ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Wicken, era, ethan.furman, lukasz.langa, ncoghlan, pmoody, xiang.zhang
Priority: normal Keywords: patch

Created on 2016-11-01 07:02 by era, last changed 2021-04-26 19:57 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 18757 merged Wicken, 2020-03-02 23:49
PR 25532 closed Wicken, 2021-04-22 19:59
PR 25533 closed Wicken, 2021-04-22 20:01
PR 25536 merged Wicken, 2021-04-22 20:05
Messages (15)
msg279855 - (view) Author: (era) Date: 2016-11-01 07:02
I would expect the following code to return ['10.9.8.8'] but it returns an empty list.

    yosemite-osx$ python3
    Python 3.5.1 (default, Dec 26 2015, 18:08:53) 
    [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import ipaddress
    >>> list(ipaddress.ip_network('10.9.8.7/32').hosts())
    []

This seems to happen for every /32 address.  I'm guessing the logic which wants to exclude the gateway and broadcast addresses from a block should treat a /32 as a special case.

I tried to look for a previous bug submission but I could not find one.  As such, it seems peculiar if this has not been reported before.  Is this actually expected behavior by some rule I am overlooking?

I tested on Linux 3.4 and OSX Yosemite Homebrew / Python 3.5.1.
msg279856 - (view) Author: (era) Date: 2016-11-01 07:04
(Meh, silly typo, of course the expected output is ['10.9.8.7'], sorry about that!)
msg279858 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-01 07:16
hosts() won't return the network address itself and the network broadcast address. So for 10.9.8.7/32, it should return [].
msg279859 - (view) Author: (era) Date: 2016-11-01 08:16
@xiang.zhang thanks for the quick reply.

I find this behavior surprising.  If I process a list of addresses, like

    ips = (
     '10.9.8.7/32'
     '10.11.12.8/28'
    )

    for test in ['10.9.8.7', '10.11.12.10']:
      if test in [str(y) for x in ips for y in ipaddress.ip_network(x).hosts()]:
        print('{0} found'.format(test))
      else:
        print('{0} not found'.format(test))

I would expect both addresses to print "found", but that's not how the current implementation works.

I agree that the /28 should not include the gateway and broadcast addresses, but I would not expect the explicitly listed /32 address to completely disappear from the output.

Are my expectations incorrect?  For code like this, what should I use instead, if not hosts()?
msg279860 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-01 09:03
I am not sure. Actually there is a special case for mask 31, you can see #27863. Its result includes both the network and broadcast address. Add Nick to see his opinion.

FYI, ipaddress (ipaddr in Py2) always return empty for 32. But there is other library returning network address for 32, for example netaddr.
msg279862 - (view) Author: (era) Date: 2016-11-01 09:31
Quick googling did not turn up anything like a credible authoritative reference for this, but in actual practice, I have seen /32 used to designate a single individual IP address in CIDR notation quite a lot.

I can see roughly three options:

  1. Status quo.  Silently surprise users who expect this to work.
  2. Silently fix.  Hard-code /32 to return a range of one IP address.
  3. Let users choose.  Similarly to the "strict=True" keyword argument in the constructor method, the code could allow for either lenient or strict semantics.

By the by, I don't see how the bug you linked to is relevant here, did you mistype the bug number?  #27863 is about _elementtree
msg279865 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-11-01 09:41
Sorry, it's #27683.
msg279867 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-11-01 10:03
I think this is an area where ipaddress just inherited ipaddr's behaviour without challenging it.

As Peter isn't currently particularly active, and I stepped back from ipaddress maintenance some time ago (since I don't work heavily enough with raw IP addresses to have the right design instincts to arbitrate edge cases like this), I'd suggest raising both this and #27683 on python-dev, pointing out that we used to have the "/31" special case (treating it the same as "/30") and inadvertently lost it in some other refactoring, but have never special cased "/32".
msg314183 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-03-21 02:13
I fixed #27683 since it looks more like an oversight and regression to me instead of a deliberate change. I'd like the behaviour to be consistent across versions. As for "/32", it needs discussion or some expert makes a choice.
msg355956 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2019-11-04 15:59
I came across this /32 issue today trying to iterate over the hosts in 127.0.0.1/32.  I think it's fair to say that any /32 network has precisely one host, and that host should by returned by IPv4Network().hosts().
msg363221 - (view) Author: Pete Wicken (Wicken) * Date: 2020-03-02 23:57
I've had a go at implementing this. I did not implement for IPv6 as this was not mentioned here, but it seems like it would make sense for it as well. I can add that in too if you like.
msg363249 - (view) Author: Pete Wicken (Wicken) * Date: 2020-03-03 09:21
Ok it was bugging me that they were different, so I also added the same logic for IPv6Networks.
msg363778 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2020-03-09 22:33
New changeset 8e9c47a947954c997d4b725f4551d50a1d896722 by Pete Wicken in branch 'master':
bpo-28577: Special case added to IP v4 and v6 hosts for /32 and /128 networks (GH-18757)
https://github.com/python/cpython/commit/8e9c47a947954c997d4b725f4551d50a1d896722
msg368638 - (view) Author: Pete Wicken (Wicken) * Date: 2020-05-11 18:12
The patch for this has been merged - I guess this can be closed now?
msg391983 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-04-26 19:57
New changeset 10ad7eb2ef61a2ba99341c804c8c960e4f284621 by Pete Wicken in branch '3.8':
[3.8] bpo-28577: Special case added to IP v4 and v6 hosts for /32 and /128 networks (GH-18757) (#25536)
https://github.com/python/cpython/commit/10ad7eb2ef61a2ba99341c804c8c960e4f284621
History
Date User Action Args
2021-04-26 19:57:07lukasz.langasetnosy: + lukasz.langa
messages: + msg391983
2021-04-26 19:56:19lukasz.langasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-04-22 20:05:06Wickensetpull_requests: + pull_request24255
2021-04-22 20:01:51Wickensetpull_requests: + pull_request24252
2021-04-22 19:59:34Wickensetpull_requests: + pull_request24251
2020-05-11 18:12:17Wickensetmessages: + msg368638
2020-03-09 22:35:17ethan.furmansetversions: + Python 3.7
2020-03-09 22:33:52ethan.furmansetmessages: + msg363778
2020-03-03 09:21:22Wickensetmessages: + msg363249
2020-03-02 23:57:49Wickensetmessages: + msg363221
2020-03-02 23:49:57Wickensetkeywords: + patch
nosy: + Wicken

pull_requests: + pull_request18112
stage: patch review
2019-11-04 15:59:13ethan.furmansetnosy: + ethan.furman
messages: + msg355956
2018-03-21 02:14:00xiang.zhangsetresolution: not a bug -> (no value)
stage: resolved -> (no value)
messages: + msg314183
versions: + Python 3.8, - Python 3.7
2016-11-01 10:03:58ncoghlansetnosy: + pmoody
messages: + msg279867
2016-11-01 09:41:28xiang.zhangsetmessages: + msg279865
2016-11-01 09:31:42erasetmessages: + msg279862
2016-11-01 09:03:15xiang.zhangsetstatus: closed -> open
versions: + Python 3.7, - Python 3.4, Python 3.5
nosy: + ncoghlan

messages: + msg279860

type: behavior
2016-11-01 08:16:05erasetmessages: + msg279859
2016-11-01 07:16:34xiang.zhangsetstatus: open -> closed

nosy: + xiang.zhang
messages: + msg279858

resolution: not a bug
stage: resolved
2016-11-01 07:04:44erasetmessages: + msg279856
2016-11-01 07:02:04eracreate