Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32 #72763

Closed
era mannequin opened this issue Nov 1, 2016 · 15 comments
Closed

ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32 #72763

era mannequin opened this issue Nov 1, 2016 · 15 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@era
Copy link
Mannequin

era mannequin commented Nov 1, 2016

BPO 28577
Nosy @ncoghlan, @ethanfurman, @ambv, @zhangyangyu, @JamoBox
PRs
  • bpo-28577: Special case added to IP v4 and v6 hosts for /32 and /128 networks #18757
  • [3.6] bpo-28577: Special case added to IP v4 and v6 hosts for /32 and… #25532
  • [3.7] bpo-28577: Special case added to IP v4 and v6 hosts for /32 and… #25533
  • [3.8] bpo-28577: Special case added to IP v4 and v6 hosts for /32 and… #25536
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-04-26.19:56:19.152>
    created_at = <Date 2016-11-01.07:02:04.765>
    labels = ['3.7', '3.8', 'type-bug', 'library']
    title = 'ipaddress.ip_network(...).hosts() returns nothing for an IPv4 /32'
    updated_at = <Date 2021-04-26.19:57:07.875>
    user = 'https://bugs.python.org/era'

    bugs.python.org fields:

    activity = <Date 2021-04-26.19:57:07.875>
    actor = 'lukasz.langa'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-04-26.19:56:19.152>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2016-11-01.07:02:04.765>
    creator = 'era'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 28577
    keywords = ['patch']
    message_count = 15.0
    messages = ['279855', '279856', '279858', '279859', '279860', '279862', '279865', '279867', '314183', '355956', '363221', '363249', '363778', '368638', '391983']
    nosy_count = 7.0
    nosy_names = ['ncoghlan', 'pmoody', 'ethan.furman', 'lukasz.langa', 'era', 'xiang.zhang', 'Wicken']
    pr_nums = ['18757', '25532', '25533', '25536']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28577'
    versions = ['Python 3.7', 'Python 3.8']

    @era
    Copy link
    Mannequin Author

    era mannequin commented Nov 1, 2016

    I would expect the following code to return ['10.9.8.8'] but it returns an empty list.

        yosemite-osx$ python3
        Python 3.5.1 (default, Dec 26 2015, 18:08:53) 
        [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
        Type "help", "copyright", "credits" or "license" for more information.
        >>> import ipaddress
        >>> list(ipaddress.ip_network('10.9.8.7/32').hosts())
        []

    This seems to happen for every /32 address. I'm guessing the logic which wants to exclude the gateway and broadcast addresses from a block should treat a /32 as a special case.

    I tried to look for a previous bug submission but I could not find one. As such, it seems peculiar if this has not been reported before. Is this actually expected behavior by some rule I am overlooking?

    I tested on Linux 3.4 and OSX Yosemite Homebrew / Python 3.5.1.

    @era era mannequin added the stdlib Python modules in the Lib dir label Nov 1, 2016
    @era
    Copy link
    Mannequin Author

    era mannequin commented Nov 1, 2016

    (Meh, silly typo, of course the expected output is ['10.9.8.7'], sorry about that!)

    @zhangyangyu
    Copy link
    Member

    hosts() won't return the network address itself and the network broadcast address. So for 10.9.8.7/32, it should return [].

    @era
    Copy link
    Mannequin Author

    era mannequin commented Nov 1, 2016

    @xiang.zhang thanks for the quick reply.

    I find this behavior surprising. If I process a list of addresses, like

        ips = (
         '10.9.8.7/32'
         '10.11.12.8/28'
        )
    
        for test in ['10.9.8.7', '10.11.12.10']:
          if test in [str(y) for x in ips for y in ipaddress.ip_network(x).hosts()]:
            print('{0} found'.format(test))
          else:
            print('{0} not found'.format(test))

    I would expect both addresses to print "found", but that's not how the current implementation works.

    I agree that the /28 should not include the gateway and broadcast addresses, but I would not expect the explicitly listed /32 address to completely disappear from the output.

    Are my expectations incorrect? For code like this, what should I use instead, if not hosts()?

    @zhangyangyu
    Copy link
    Member

    I am not sure. Actually there is a special case for mask 31, you can see bpo-27863. Its result includes both the network and broadcast address. Add Nick to see his opinion.

    FYI, ipaddress (ipaddr in Py2) always return empty for 32. But there is other library returning network address for 32, for example netaddr.

    @zhangyangyu zhangyangyu added the 3.7 (EOL) end of life label Nov 1, 2016
    @zhangyangyu zhangyangyu reopened this Nov 1, 2016
    @zhangyangyu zhangyangyu added the type-bug An unexpected behavior, bug, or error label Nov 1, 2016
    @era
    Copy link
    Mannequin Author

    era mannequin commented Nov 1, 2016

    Quick googling did not turn up anything like a credible authoritative reference for this, but in actual practice, I have seen /32 used to designate a single individual IP address in CIDR notation quite a lot.

    I can see roughly three options:

    1. Status quo. Silently surprise users who expect this to work.
    2. Silently fix. Hard-code /32 to return a range of one IP address.
    3. Let users choose. Similarly to the "strict=True" keyword argument in the constructor method, the code could allow for either lenient or strict semantics.

    By the by, I don't see how the bug you linked to is relevant here, did you mistype the bug number? bpo-27863 is about _elementtree

    @zhangyangyu
    Copy link
    Member

    Sorry, it's bpo-27683.

    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Nov 1, 2016

    I think this is an area where ipaddress just inherited ipaddr's behaviour without challenging it.

    As Peter isn't currently particularly active, and I stepped back from ipaddress maintenance some time ago (since I don't work heavily enough with raw IP addresses to have the right design instincts to arbitrate edge cases like this), I'd suggest raising both this and bpo-27683 on python-dev, pointing out that we used to have the "/31" special case (treating it the same as "/30") and inadvertently lost it in some other refactoring, but have never special cased "/32".

    @zhangyangyu
    Copy link
    Member

    I fixed bpo-27683 since it looks more like an oversight and regression to me instead of a deliberate change. I'd like the behaviour to be consistent across versions. As for "/32", it needs discussion or some expert makes a choice.

    @zhangyangyu zhangyangyu added 3.8 only security fixes and removed 3.7 (EOL) end of life invalid labels Mar 21, 2018
    @ethanfurman
    Copy link
    Member

    I came across this /32 issue today trying to iterate over the hosts in 127.0.0.1/32. I think it's fair to say that any /32 network has precisely one host, and that host should by returned by IPv4Network().hosts().

    @JamoBox
    Copy link
    Mannequin

    JamoBox mannequin commented Mar 2, 2020

    I've had a go at implementing this. I did not implement for IPv6 as this was not mentioned here, but it seems like it would make sense for it as well. I can add that in too if you like.

    @JamoBox
    Copy link
    Mannequin

    JamoBox mannequin commented Mar 3, 2020

    Ok it was bugging me that they were different, so I also added the same logic for IPv6Networks.

    @ethanfurman
    Copy link
    Member

    New changeset 8e9c47a by Pete Wicken in branch 'master':
    bpo-28577: Special case added to IP v4 and v6 hosts for /32 and /128 networks (GH-18757)
    8e9c47a

    @ethanfurman ethanfurman added the 3.7 (EOL) end of life label Mar 9, 2020
    @JamoBox
    Copy link
    Mannequin

    JamoBox mannequin commented May 11, 2020

    The patch for this has been merged - I guess this can be closed now?

    @ambv ambv closed this as completed Apr 26, 2021
    @ambv
    Copy link
    Contributor

    ambv commented Apr 26, 2021

    New changeset 10ad7eb by Pete Wicken in branch '3.8':
    [3.8] bpo-28577: Special case added to IP v4 and v6 hosts for /32 and /128 networks (GH-18757) (bpo-25536)
    10ad7eb

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants