Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

optimize v4 & v6 netmask parsing #65685

Closed
pitrou opened this issue May 12, 2014 · 8 comments
Closed

optimize v4 & v6 netmask parsing #65685

pitrou opened this issue May 12, 2014 · 8 comments
Labels
performance Performance or resource usage stdlib Python modules in the Lib dir

Comments

@pitrou
Copy link
Member

pitrou commented May 12, 2014

BPO 21486
Nosy @ncoghlan, @pitrou, @serhiy-storchaka
Dependencies
  • bpo-16531: Allow IPNetwork to take a tuple
    		•
    Files
  • v4_netmask.patch
  • netmask.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2014-05-15.19:06:11.217>
created_at = <Date 2014-05-12.18:24:45.892>
labels = ['library', 'performance']
title = 'optimize v4 & v6 netmask parsing'
updated_at = <Date 2014-05-15.19:06:11.216>
user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2014-05-15.19:06:11.216>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-05-15.19:06:11.217>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2014-05-12.18:24:45.892>
creator = 'pitrou'
dependencies = ['16531']
files = ['35231', '35232']
hgrepos = []
issue_num = 21486
keywords = ['patch']
message_count = 8.0
messages = ['218344', '218349', '218481', '218482', '218504', '218609', '218610', '218621']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'pitrou', 'pmoody', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue21486'
versions = ['Python 3.5']
    @pitrou
    Copy link
    Member Author

    pitrou commented May 12, 2014

    Here is a patch to optimize ipv4 netmask parsing by maintaining a cache (there are not many valid ipv4 netmask representations). This should be especially useful with bpo-16531.

    @pitrou pitrou added stdlib Python modules in the Lib dir performance Performance or resource usage labels May 12, 2014
    @pitrou
    Copy link
    Member Author

    pitrou commented May 12, 2014

    Updated patch, also optimizing v6 netmask parsing (same principle).

    Before patch:

    $ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv6Network(('2001:db8::', 96))"
10000 loops, best of 3: 26.1 usec per loop

$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv4Network(('10.0.0.0', 23))"
100000 loops, best of 3: 17 usec per loop

    After patch:

    $ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv6Network(('2001:db8::', 96))"
100000 loops, best of 3: 13.8 usec per loop

$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv4Network(('10.0.0.0', 23))"
100000 loops, best of 3: 14.3 usec per loop

    @pitrou pitrou changed the title optimize v4 netmask parsing optimize v4 & v6 netmask parsing May 12, 2014
    @serhiy-storchaka
    Copy link
    Member

    Why not just use functools.lru_cache?

    @pitrou
    Copy link
    Member Author

    pitrou commented May 13, 2014

    Because that would incur the cost of LRU logic and locking, which we don't need here.

    @serhiy-storchaka
    Copy link
    Member

    With C implementation (bpo-14373) functools.lru_cache is so fast as manually written specialized code.

    @serhiy-storchaka
    Copy link
    Member

    What I want to say, the patch LGTM, but after committing bpo-14373 we should simplify the code by using functools.lru_cache().

    @pitrou
    Copy link
    Member Author

    pitrou commented May 15, 2014

    Actually, using lru_cache(maxsize=None) would enable a simple infinite cache like in the patch. But it's not like a lot of code would be saved.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented May 15, 2014

    New changeset 2158614e1607 by Antoine Pitrou in branch 'default':
    Issue bpo-21486: Optimize parsing of netmasks in ipaddress.IPv4Network and ipaddress.IPv6Network.
    http://hg.python.org/cpython/rev/2158614e1607

    @pitrou pitrou closed this as completed May 15, 2014
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    performance Performance or resource usage stdlib Python modules in the Lib dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @pitrou @serhiy-storchaka