diff -r 2e9cba1d1554 -r f5c57ba1124b Doc/howto/ipaddress.rst --- a/Doc/howto/ipaddress.rst Sat Jul 07 23:05:59 2012 +1000 +++ b/Doc/howto/ipaddress.rst Sat Jul 07 15:32:29 2012 +0200 @@ -9,11 +9,11 @@ .. topic:: Overview - This document aims to provide a gentle introduction to the - :mod:`ipaddress` module. It is aimed primarily at users that aren't - already familiar with IP networking terminology, but may also be useful - to network engineers wanting an overview of how :mod:`ipaddress` - represents IP network addressing concepts. + This document aims to provide a gentle introduction to :mod:`ipaddress` + module. It is aimed primarily at users that aren't already familiar with + IP networking terminology, but may also be useful to network engineers + wanting an overview of how the ipaddress module represents IP network + addressing concepts. Creating Address/Network/Interface objects @@ -45,9 +45,8 @@ Addresses, often referred to as "host addresses" are the most basic unit when working with IP addressing. The simplest way to create addresses is -to use the :func:`ipaddress.ip_address` factory function, which automatically -determines whether to create an IPv4 or IPv6 address based on the passed in -value:: +to use the :func:`ipaddress.ip_address` factory function, which automatically determines +whether to create an IPv4 or IPv6 address based on the passed in value:: >>> ipaddress.ip_address('192.0.2.1') IPv4Address('192.0.2.1') @@ -122,9 +121,8 @@ >>> ipaddress.ip_network(42540766411282592856903984951653826560) IPv6Network('2001:db8::/128') -As with addresses, creation of a particular kind of network can be forced -by calling the class constructor directly instead of using the factory -function. +Creation of a particular kind of network can be forced by calling the +class constructor directly instead of using the factory function. Host Interfaces @@ -132,7 +130,7 @@ As mentioned just above, if you need to describe an address on a particular network, neither the address nor the network classes are sufficient. -Notation like ``192.0.2.1/24`` is commonly used by network engineers and the +Notation like ``192.0.2.1/24`` is commonly used network engineers and the people who write tools for firewalls and routers as shorthand for "the host ``192.0.2.1`` on the network ``192.0.2.0/24``", Accordingly, :mod:`ipaddress` provides a set of hybrid classes that associate an address with a particular @@ -215,19 +213,10 @@ Exploding or compressing the address:: + >>> net6.exploded + '2001:0000:0000:0000:0000:0000:0000:0000/96' >>> addr6.exploded - '2001:0db8:0000:0000:0000:0000:0000:0000' - >>> addr6.compressed - '2001:db8::' - >>> net6.exploded - '2001:0db8:0000:0000:0000:0000:0000:0000/96' - >>> net6.compressed - '2001:db8::/96' - -While IPv4 doesn't support explosion or compression, the associated objects -still provide the relevant properties so that version neutral code can -easily ensure the most concise or most verbose form is used for IPv6 -addresses while still correctly handling IPv4 addresses. + '2001:0000:0000:0000:0000:0000:0000:0001' Networks as lists of Addresses @@ -288,49 +277,23 @@ 3221225985 -Getting more detail when instance creation fails -================================================ +Exceptions raised by :mod:`ipaddress` +===================================== When creating address/network/interface objects using the version-agnostic -factory functions, any errors will be reported as :exc:`ValueError` with -a generic error message that simply says the passed in value was not -recognised as an object of that type. The lack of a specific error is -because it's necessary to know whether the value is *supposed* to be IPv4 -or IPv6 in order to provide more detail on why it has been rejected. +factory functions, any errors will be reported as :exc:`ValueError`. -To support use cases where it is useful to have access to this additional -detail, the individual class constructors actually raise the -:exc:`ValueError` subclasses :exc:`ipaddress.AddressValueError` and -:exc:`ipaddress.NetmaskValueError` to indicate exactly which part of -the definition failed to parse correctly. +For some use cases, it desirable to know whether it is the address or the +netmask which is incorrect. To support these use cases, the class +constructors actually raise the :exc:`ValueError` subclasses +:exc:`ipaddress.AddressValueError` and :exc:`ipaddress.NetmaskValueError` +to indicate exactly which part of the definition failed to parse correctly. -The error messages are significantly more detailed when using the -class constructors directly. For example:: - - >>> ipaddress.ip_address("192.168.0.256") - Traceback (most recent call last): - ... - ValueError: '192.168.0.256' does not appear to be an IPv4 or IPv6 address - >>> ipaddress.IPv4Address("192.168.0.256") - Traceback (most recent call last): - ... - ipaddress.AddressValueError: Octet 256 (> 255) not permitted in '192.168.0.256' - - >>> ipaddress.ip_network("192.168.0.1/64") - Traceback (most recent call last): - ... - ValueError: '192.168.0.1/64' does not appear to be an IPv4 or IPv6 network - >>> ipaddress.IPv4Network("192.168.0.1/64") - Traceback (most recent call last): - ... - ipaddress.NetmaskValueError: '64' is not a valid netmask - -However, both of the module specific exceptions have :exc:`ValueError` as their +Both of the module specific exceptions have :exc:`ValueError` as their parent class, so if you're not concerned with the particular type of error, you can still write code like the following:: try: - network = ipaddress.IPv4Network(address) + ipaddress.IPv4Address(address) except ValueError: - print('address/netmask is invalid for IPv4:', address) - + print('address/netmask is invalid:', address) diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/importlib/_bootstrap.py --- a/Lib/importlib/_bootstrap.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/importlib/_bootstrap.py Sat Jul 07 15:32:29 2012 +0200 @@ -140,7 +140,7 @@ def _wrap(new, old): - """Simple substitute for functools.update_wrapper.""" + """Simple substitute for functools.wraps.""" for replace in ['__module__', '__name__', '__qualname__', '__doc__']: if hasattr(old, replace): setattr(new, replace, getattr(old, replace)) @@ -345,7 +345,7 @@ """Set __package__ on the returned module.""" def set_package_wrapper(*args, **kwargs): module = fxn(*args, **kwargs) - if getattr(module, '__package__', None) is None: + if not hasattr(module, '__package__') or module.__package__ is None: module.__package__ = module.__name__ if not hasattr(module, '__path__'): module.__package__ = module.__package__.rpartition('.')[0] @@ -438,7 +438,7 @@ """Decorator to verify the named module is built-in.""" def _requires_builtin_wrapper(self, fullname): if fullname not in sys.builtin_module_names: - raise ImportError("{} is not a built-in module".format(fullname), + raise ImportError("{0} is not a built-in module".format(fullname), name=fullname) return fxn(self, fullname) _wrap(_requires_builtin_wrapper, fxn) @@ -449,7 +449,7 @@ """Decorator to verify the named module is frozen.""" def _requires_frozen_wrapper(self, fullname): if not _imp.is_frozen(fullname): - raise ImportError("{} is not a frozen module".format(fullname), + raise ImportError("{0} is not a frozen module".format(fullname), name=fullname) return fxn(self, fullname) _wrap(_requires_frozen_wrapper, fxn) @@ -511,7 +511,7 @@ @classmethod @_requires_builtin def is_package(cls, fullname): - """Return False as built-in modules are never packages.""" + """Return None as built-in modules are never packages.""" return False @@ -936,7 +936,7 @@ return len(self._recalculate()) def __repr__(self): - return "_NamespacePath({!r})".format(self._path) + return "_NamespacePath({0!r})".format(self._path) def __contains__(self, item): return item in self._recalculate() @@ -1198,7 +1198,7 @@ if len(bits) < level: raise ValueError('attempted relative import beyond top-level package') base = bits[0] - return '{}.{}'.format(base, name) if name else base + return '{0}.{1}'.format(base, name) if name else base def _find_module(name, path): @@ -1228,7 +1228,7 @@ if not isinstance(package, str): raise TypeError("__package__ not set to a string") elif package not in sys.modules: - msg = ("Parent module {!r} not loaded, cannot perform relative " + msg = ("Parent module {0!r} not loaded, cannot perform relative " "import") raise SystemError(msg.format(package)) if not name and level == 0: @@ -1267,7 +1267,7 @@ parent_module = sys.modules[parent] setattr(parent_module, name.rpartition('.')[2], module) # Set __package__ if the loader did not. - if getattr(module, '__package__', None) is None: + if not hasattr(module, '__package__') or module.__package__ is None: try: module.__package__ = module.__name__ if not hasattr(module, '__path__'): @@ -1336,12 +1336,11 @@ fromlist = list(fromlist) fromlist.remove('*') fromlist.extend(module.__all__) - for x in fromlist: - if not hasattr(module, x): - try: - import_('{}.{}'.format(module.__name__, x)) - except ImportError: - pass + for x in (y for y in fromlist if not hasattr(module, y)): + try: + import_('{0}.{1}'.format(module.__name__, x)) + except ImportError: + pass return module diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/ipaddress.py --- a/Lib/ipaddress.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/ipaddress.py Sat Jul 07 15:32:29 2012 +0200 @@ -17,6 +17,7 @@ IPV4LENGTH = 32 IPV6LENGTH = 128 + class AddressValueError(ValueError): """A Value Error related to the address.""" @@ -116,7 +117,7 @@ except (AddressValueError, NetmaskValueError): pass - raise ValueError('%r does not appear to be an IPv4 or IPv6 interface' % + raise ValueError('%r does not appear to be an IPv4 or IPv6 network' % address) @@ -156,13 +157,6 @@ raise ValueError("Address negative or too large for IPv6") -def _split_optional_netmask(address): - """Helper to split the netmask and raise AddressValueError if needed""" - addr = str(address).split('/') - if len(addr) > 2: - raise AddressValueError("Only one '/' permitted in %r" % address) - return addr - def _find_address_range(addresses): """Find a sequence of IPv#Address. @@ -214,10 +208,8 @@ if number == 0: return bits for i in range(bits): - if (number >> i) & 1: + if (number >> i) % 2: return i - # All bits of interest were zero, even if there are more in the number - return bits def summarize_address_range(first, last): @@ -265,13 +257,20 @@ first_int = first._ip last_int = last._ip while first_int <= last_int: - nbits = min(_count_righthand_zero_bits(first_int, ip_bits), - (last_int - first_int + 1).bit_length() - 1) - net = ip('%s/%d' % (first, ip_bits - nbits)) + nbits = _count_righthand_zero_bits(first_int, ip_bits) + current = None + while nbits >= 0: + addend = 2**nbits - 1 + current = first_int + addend + nbits -= 1 + if current <= last_int: + break + prefix = _get_prefix_length(first_int, current, ip_bits) + net = ip('%s/%d' % (first, prefix)) yield net - first_int += 1 << nbits - if first_int - 1 == ip._ALL_ONES: + if current == ip._ALL_ONES: break + first_int = current + 1 first = first.__class__(first_int) @@ -299,28 +298,26 @@ passed. """ - while True: - last_addr = None - ret_array = [] - optimized = False + ret_array = [] + optimized = False - for cur_addr in addresses: - if not ret_array: - last_addr = cur_addr - ret_array.append(cur_addr) - elif (cur_addr.network_address >= last_addr.network_address and - cur_addr.broadcast_address <= last_addr.broadcast_address): - optimized = True - elif cur_addr == list(last_addr.supernet().subnets())[1]: - ret_array[-1] = last_addr = last_addr.supernet() - optimized = True - else: - last_addr = cur_addr - ret_array.append(cur_addr) + for cur_addr in addresses: + if not ret_array: + ret_array.append(cur_addr) + continue + if (cur_addr.network_address >= ret_array[-1].network_address and + cur_addr.broadcast_address <= ret_array[-1].broadcast_address): + optimized = True + elif cur_addr == list(ret_array[-1].supernet().subnets())[1]: + ret_array.append(ret_array.pop().supernet()) + optimized = True + else: + ret_array.append(cur_addr) - addresses = ret_array - if not optimized: - return addresses + if optimized: + return _collapse_addresses_recursive(ret_array) + + return ret_array def collapse_addresses(addresses): @@ -449,7 +446,13 @@ An integer, the prefix length. """ - return mask - _count_righthand_zero_bits(ip_int, mask) + while mask: + if ip_int & 1 == 1: + break + ip_int >>= 1 + mask -= 1 + + return mask def _ip_string_from_prefix(self, prefixlen=None): """Turn a prefix length into a dotted decimal string. @@ -478,7 +481,7 @@ def __init__(self, address): if (not isinstance(address, bytes) and '/' in str(address)): - raise AddressValueError("Unexpected '/' in %r" % address) + raise AddressValueError(address) def __index__(self): return self._ip @@ -588,16 +591,18 @@ or broadcast addresses. """ - network = int(self.network_address) - broadcast = int(self.broadcast_address) - for x in range(network + 1, broadcast): - yield self._address_class(x) + cur = int(self.network_address) + 1 + bcast = int(self.broadcast_address) - 1 + while cur <= bcast: + cur += 1 + yield self._address_class(cur - 1) def __iter__(self): - network = int(self.network_address) - broadcast = int(self.broadcast_address) - for x in range(network, broadcast + 1): - yield self._address_class(x) + cur = int(self.network_address) + bcast = int(self.broadcast_address) + while cur <= bcast: + cur += 1 + yield self._address_class(cur - 1) def __getitem__(self, n): network = int(self.network_address) @@ -651,12 +656,12 @@ return not lt def __eq__(self, other): - try: - return (self._version == other._version and - self.network_address == other.network_address and - int(self.netmask) == int(other.netmask)) - except AttributeError: - return NotImplemented + if not isinstance(other, _BaseNetwork): + raise TypeError('%s and %s are not of the same type' % ( + self, other)) + return (self._version == other._version and + self.network_address == other.network_address and + int(self.netmask) == int(other.netmask)) def __ne__(self, other): eq = self.__eq__(other) @@ -987,7 +992,7 @@ _DECIMAL_DIGITS = frozenset('0123456789') # the valid octets for host and netmasks. only useful for IPv4. - _valid_mask_octets = frozenset((255, 254, 252, 248, 240, 224, 192, 128, 0)) + _valid_mask_octets = set((255, 254, 252, 248, 240, 224, 192, 128, 0)) def __init__(self, address): self._version = 4 @@ -1009,17 +1014,17 @@ AddressValueError: if ip_str isn't a valid IPv4 Address. """ - if not ip_str: - raise AddressValueError('Address cannot be empty') - octets = ip_str.split('.') if len(octets) != 4: - raise AddressValueError("Expected 4 octets in %r" % ip_str) + raise AddressValueError(ip_str) - try: - return int.from_bytes(map(self._parse_octet, octets), 'big') - except ValueError as exc: - raise AddressValueError("%s in %r" % (exc, ip_str)) from None + packed_ip = 0 + for oc in octets: + try: + packed_ip = (packed_ip << 8) | self._parse_octet(oc) + except ValueError: + raise AddressValueError(ip_str) from None + return packed_ip def _parse_octet(self, octet_str): """Convert a decimal octet into an integer. @@ -1034,21 +1039,15 @@ ValueError: if the octet isn't strictly a decimal from [0..255]. """ - if not octet_str: - raise ValueError("Empty octet not permitted") # Whitelist the characters, since int() allows a lot of bizarre stuff. + # Higher level wrappers convert these to more informative errors if not self._DECIMAL_DIGITS.issuperset(octet_str): - raise ValueError("Only decimal digits permitted in %r" % octet_str) - # Convert to integer (we know digits are legal) + raise ValueError octet_int = int(octet_str, 10) - # Any octets that look like they *might* be written in octal, - # and which don't look exactly the same in both octal and - # decimal are rejected as ambiguous - if octet_int > 7 and octet_str[0] == '0': - raise ValueError("Ambiguous leading zero in %r not permitted" % - octet_str) - if octet_int > 255: - raise ValueError("Octet %d (> 255) not permitted" % octet_int) + # Disallow leading zeroes, because no clear standard exists on + # whether these should be interpreted as decimal or octal. + if octet_int > 255 or (octet_str[0] == '0' and len(octet_str) > 1): + raise ValueError return octet_int def _string_from_ip_int(self, ip_int): @@ -1061,7 +1060,11 @@ The IP address as a string in dotted decimal notation. """ - return '.'.join(map(str, ip_int.to_bytes(4, 'big'))) + octets = [] + for _ in range(4): + octets.insert(0, str(ip_int & 0xFF)) + ip_int >>= 8 + return '.'.join(octets) def _is_valid_netmask(self, netmask): """Verify that the netmask is valid. @@ -1077,16 +1080,11 @@ """ mask = netmask.split('.') if len(mask) == 4: - try: - for x in mask: - if int(x) not in self._valid_mask_octets: - return False - except ValueError: - # Found something that isn't an integer or isn't valid + if [x for x in mask if int(x) not in self._valid_mask_octets]: return False - for idx, y in enumerate(mask): - if idx > 0 and y > mask[idx - 1]: - return False + if [y for idx, y in enumerate(mask) if idx > 0 and + y > mask[idx - 1]]: + return False return True try: netmask = int(netmask) @@ -1106,7 +1104,7 @@ """ bits = ip_str.split('.') try: - parts = [x for x in map(int, bits) if x in self._valid_mask_octets] + parts = [int(x) for x in bits if int(x) in self._valid_mask_octets] except ValueError: return False if len(parts) != len(bits): @@ -1253,8 +1251,7 @@ # Constructing from a packed address if isinstance(address, bytes): if len(address) != 4: - msg = "Packed address %r must be exactly 4 bytes" - raise AddressValueError(msg % address) + raise AddressValueError(address) self._ip = struct.unpack('!I', address)[0] return @@ -1278,7 +1275,9 @@ self._prefixlen = self._max_prefixlen return - addr = _split_optional_netmask(address) + addr = str(address).split('/') + if len(addr) > 2: + raise AddressValueError(address) IPv4Address.__init__(self, addr[0]) self.network = IPv4Network(address, strict=False) @@ -1383,8 +1382,7 @@ # Constructing from a packed address if isinstance(address, bytes): if len(address) != 4: - msg = "Packed address %r must be exactly 4 bytes" - raise AddressValueError(msg % address) + raise AddressValueError(address) self.network_address = IPv4Address( struct.unpack('!I', address)[0]) self._prefixlen = self._max_prefixlen @@ -1404,9 +1402,12 @@ # Assume input argument to be string or any object representation # which converts into a formatted IP prefix string. - addr = _split_optional_netmask(address) + addr = str(address).split('/') self.network_address = IPv4Address(self._ip_int_from_string(addr[0])) + if len(addr) > 2: + raise AddressValueError(address) + if len(addr) == 2: mask = addr[1].split('.') @@ -1419,15 +1420,14 @@ self.netmask = IPv4Address( self._ip_int_from_string(addr[1]) ^ self._ALL_ONES) else: - raise NetmaskValueError('%r is not a valid netmask' + raise NetmaskValueError('%s is not a valid netmask' % addr[1]) self._prefixlen = self._prefix_from_ip_int(int(self.netmask)) else: # We have a netmask in prefix length form. if not self._is_valid_netmask(addr[1]): - raise NetmaskValueError('%r is not a valid netmask' - % addr[1]) + raise NetmaskValueError(addr[1]) self._prefixlen = int(addr[1]) self.netmask = IPv4Address(self._ip_int_from_prefix( self._prefixlen)) @@ -1477,44 +1477,31 @@ AddressValueError: if ip_str isn't a valid IPv6 Address. """ - if not ip_str: - raise AddressValueError('Address cannot be empty') - parts = ip_str.split(':') # An IPv6 address needs at least 2 colons (3 parts). - _min_parts = 3 - if len(parts) < _min_parts: - msg = "At least %d parts expected in %r" % (_min_parts, ip_str) - raise AddressValueError(msg) + if len(parts) < 3: + raise AddressValueError(ip_str) # If the address has an IPv4-style suffix, convert it to hexadecimal. if '.' in parts[-1]: - try: - ipv4_int = IPv4Address(parts.pop())._ip - except AddressValueError as exc: - raise AddressValueError("%s in %r" % (exc, ip_str)) from None + ipv4_int = IPv4Address(parts.pop())._ip parts.append('%x' % ((ipv4_int >> 16) & 0xFFFF)) parts.append('%x' % (ipv4_int & 0xFFFF)) # An IPv6 address can't have more than 8 colons (9 parts). - # The extra colon comes from using the "::" notation for a single - # leading or trailing zero part. - _max_parts = self._HEXTET_COUNT + 1 - if len(parts) > _max_parts: - msg = "At most %d colons permitted in %r" % (_max_parts-1, ip_str) - raise AddressValueError(msg) + if len(parts) > self._HEXTET_COUNT + 1: + raise AddressValueError(ip_str) # Disregarding the endpoints, find '::' with nothing in between. # This indicates that a run of zeroes has been skipped. - skip_index = None - for i in range(1, len(parts) - 1): - if not parts[i]: - if skip_index is not None: - # Can't have more than one '::' - msg = "At most one '::' permitted in %r" % ip_str - raise AddressValueError(msg) - skip_index = i + try: + skip_index, = ( + [i for i in range(1, len(parts) - 1) if not parts[i]] or + [None]) + except ValueError: + # Can't have more than one '::' + raise AddressValueError(ip_str) from None # parts_hi is the number of parts to copy from above/before the '::' # parts_lo is the number of parts to copy from below/after the '::' @@ -1525,30 +1512,20 @@ if not parts[0]: parts_hi -= 1 if parts_hi: - msg = "Leading ':' only permitted as part of '::' in %r" - raise AddressValueError(msg % ip_str) # ^: requires ^:: + raise AddressValueError(ip_str) # ^: requires ^:: if not parts[-1]: parts_lo -= 1 if parts_lo: - msg = "Trailing ':' only permitted as part of '::' in %r" - raise AddressValueError(msg % ip_str) # :$ requires ::$ + raise AddressValueError(ip_str) # :$ requires ::$ parts_skipped = self._HEXTET_COUNT - (parts_hi + parts_lo) if parts_skipped < 1: - msg = "Expected at most %d other parts with '::' in %r" - raise AddressValueError(msg % (self._HEXTET_COUNT-1, ip_str)) + raise AddressValueError(ip_str) else: # Otherwise, allocate the entire address to parts_hi. The # endpoints could still be empty, but _parse_hextet() will check # for that. if len(parts) != self._HEXTET_COUNT: - msg = "Exactly %d parts expected without '::' in %r" - raise AddressValueError(msg % (self._HEXTET_COUNT, ip_str)) - if not parts[0]: - msg = "Leading ':' only permitted as part of '::' in %r" - raise AddressValueError(msg % ip_str) # ^: requires ^:: - if not parts[-1]: - msg = "Trailing ':' only permitted as part of '::' in %r" - raise AddressValueError(msg % ip_str) # :$ requires ::$ + raise AddressValueError(ip_str) parts_hi = len(parts) parts_lo = 0 parts_skipped = 0 @@ -1564,8 +1541,8 @@ ip_int <<= 16 ip_int |= self._parse_hextet(parts[i]) return ip_int - except ValueError as exc: - raise AddressValueError("%s in %r" % (exc, ip_str)) from None + except ValueError: + raise AddressValueError(ip_str) from None def _parse_hextet(self, hextet_str): """Convert an IPv6 hextet string into an integer. @@ -1584,15 +1561,12 @@ # Whitelist the characters, since int() allows a lot of bizarre stuff. # Higher level wrappers convert these to more informative errors if not self._HEX_DIGITS.issuperset(hextet_str): - raise ValueError("Only hex digits permitted in %r" % hextet_str) + raise ValueError if len(hextet_str) > 4: - msg = "At most 4 characters permitted in %r" - raise ValueError(msg % hextet_str) + raise ValueError hextet_int = int(hextet_str, 16) if hextet_int > 0xFFFF: - # This is unreachable due to the string length check above - msg = "Part 0x%X (> 0xFFFF) not permitted" - raise ValueError(msg % hextet_int) + raise ValueError return hextet_int def _compress_hextets(self, hextets): @@ -1662,7 +1636,9 @@ raise ValueError('IPv6 address is too large') hex_str = '%032x' % ip_int - hextets = ['%x' % int(hex_str[x:x+4], 16) for x in range(0, 32, 4)] + hextets = [] + for x in range(0, 32, 4): + hextets.append('%x' % int(hex_str[x:x+4], 16)) hextets = self._compress_hextets(hextets) return ':'.join(hextets) @@ -1685,8 +1661,11 @@ ip_str = str(self) ip_int = self._ip_int_from_string(ip_str) - hex_str = '%032x' % ip_int - parts = [hex_str[x:x+4] for x in range(0, 32, 4)] + parts = [] + for i in range(self._HEXTET_COUNT): + parts.append('%04x' % (ip_int & 0xFFFF)) + ip_int >>= 16 + parts.reverse() if isinstance(self, (_BaseNetwork, IPv6Interface)): return '%s/%d' % (':'.join(parts), self.prefixlen) return ':'.join(parts) @@ -1733,9 +1712,9 @@ IPv6Network('FE00::/9')] if isinstance(self, _BaseAddress): - return any(self in x for x in reserved_networks) - return any(self.network_address in x and self.broadcast_address in x - for x in reserved_networks) + return len([x for x in reserved_networks if self in x]) > 0 + return len([x for x in reserved_networks if self.network_address in x + and self.broadcast_address in x]) > 0 @property def is_link_local(self): @@ -1890,8 +1869,7 @@ # Constructing from a packed address if isinstance(address, bytes): if len(address) != 16: - msg = "Packed address %r must be exactly 16 bytes" - raise AddressValueError(msg % address) + raise AddressValueError(address) tmp = struct.unpack('!QQ', address) self._ip = (tmp[0] << 64) | tmp[1] return @@ -1899,6 +1877,9 @@ # Assume input argument to be string or any object representation # which converts into a formatted IP string. addr_str = str(address) + if not addr_str: + raise AddressValueError('') + self._ip = self._ip_int_from_string(addr_str) @property @@ -1916,7 +1897,7 @@ self._prefixlen = self._max_prefixlen return - addr = _split_optional_netmask(address) + addr = str(address).split('/') IPv6Address.__init__(self, addr[0]) self.network = IPv6Network(address, strict=False) self.netmask = self.network.netmask @@ -2022,8 +2003,7 @@ # Constructing from a packed address if isinstance(address, bytes): if len(address) != 16: - msg = "Packed address %r must be exactly 16 bytes" - raise AddressValueError(msg % address) + raise AddressValueError(address) tmp = struct.unpack('!QQ', address) self.network_address = IPv6Address((tmp[0] << 64) | tmp[1]) self._prefixlen = self._max_prefixlen @@ -2032,7 +2012,10 @@ # Assume input argument to be string or any object representation # which converts into a formatted IP prefix string. - addr = _split_optional_netmask(address) + addr = str(address).split('/') + + if len(addr) > 2: + raise AddressValueError(address) self.network_address = IPv6Address(self._ip_int_from_string(addr[0])) @@ -2040,8 +2023,7 @@ if self._is_valid_netmask(addr[1]): self._prefixlen = int(addr[1]) else: - raise NetmaskValueError('%r is not a valid netmask' - % addr[1]) + raise NetmaskValueError(addr[1]) else: self._prefixlen = self._max_prefixlen diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/ntpath.py --- a/Lib/ntpath.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/ntpath.py Sat Jul 07 15:32:29 2012 +0200 @@ -35,48 +35,41 @@ altsep = '/' devnull = 'nul' -def _get_empty(path): - if isinstance(path, bytes): - return b'' +# constants to be used internally so that we make the code more maintainable +_empty = '' +_sep = '\\' +_altsep = '/' +_bothsep = '\\/' +_dot = '.' +_colon = ':' +_tilde = '~' + +# quick look up to avoid calling instance a huge amounts of times +unicode_bytes_map = { + _empty: b'', + _sep: b'\\', + _altsep: b'/', + _bothsep: b'\\/', + _dot: b'.', + _colon: b':', + _tilde: b'~', +} + + +def _get_literal(constant, is_bytes): + if is_bytes: + return unicode_bytes_map[constant] else: - return '' + return constant -def _get_sep(path): - if isinstance(path, bytes): - return b'\\' - else: - return '\\' -def _get_altsep(path): - if isinstance(path, bytes): - return b'/' - else: - return '/' - -def _get_bothseps(path): - if isinstance(path, bytes): - return b'\\/' - else: - return '\\/' - -def _get_dot(path): - if isinstance(path, bytes): - return b'.' - else: - return '.' - -def _get_colon(path): - if isinstance(path, bytes): - return b':' - else: - return ':' - -def _get_special(path): - if isinstance(path, bytes): +def _get_special(path, is_bytes): + if is_bytes: return (b'\\\\.\\', b'\\\\?\\') else: return ('\\\\.\\', '\\\\?\\') + # Normalize the case of a pathname and map slashes to backslashes. # Other normalizations (such as optimizing '../' away) are not done # (this is done by normpath). @@ -88,7 +81,9 @@ if not isinstance(s, (bytes, str)): raise TypeError("normcase() argument must be str or bytes, " "not '{}'".format(s.__class__.__name__)) - return s.replace(_get_altsep(s), _get_sep(s)).lower() + is_bytes = isinstance(s, bytes) + return s.replace(_get_literal(_altsep, is_bytes), + _get_literal(_sep, is_bytes)).lower() # Return whether a path is absolute. @@ -100,7 +95,8 @@ def isabs(s): """Test whether a path is absolute""" s = splitdrive(s)[1] - return len(s) > 0 and s[:1] in _get_bothseps(s) + is_bytes = isinstance(s, bytes) + return len(s) > 0 and s[:1] in _get_literal(_bothsep, is_bytes) # Join two (or more) paths. @@ -109,9 +105,10 @@ """Join two or more pathname components, inserting "\\" as needed. If any component is an absolute path, all previous path components will be discarded.""" - sep = _get_sep(a) - seps = _get_bothseps(a) - colon = _get_colon(a) + is_bytes = isinstance(a, bytes) + sep = _get_literal(_sep, is_bytes) + seps = _get_literal(_bothsep, is_bytes) + colon = _get_literal(_colon, is_bytes) path = a for b in p: b_wins = 0 # set to 1 iff b makes path irrelevant @@ -204,9 +201,10 @@ Paths cannot contain both a drive letter and a UNC path. """ - empty = _get_empty(p) + is_bytes = isinstance(p, bytes) + empty = _get_literal(_empty, is_bytes) if len(p) > 1: - sep = _get_sep(p) + sep = _get_literal(_sep, is_bytes) normp = normcase(p) if (normp[0:2] == sep*2) and (normp[2:3] != sep): # is a UNC path: @@ -224,7 +222,7 @@ if index2 == -1: index2 = len(p) return p[:index2], p[index2:] - if normp[1:2] == _get_colon(p): + if normp[1:2] == _get_literal(_colon, is_bytes): return p[:2], p[2:] return empty, p @@ -244,7 +242,8 @@ import warnings warnings.warn("ntpath.splitunc is deprecated, use ntpath.splitdrive instead", DeprecationWarning) - sep = _get_sep(p) + is_bytes = isinstance(p, bytes) + sep = _get_literal(_sep, is_bytes) if not p[1:2]: return p[:0], p # Drive letter present firstTwo = p[0:2] @@ -276,7 +275,8 @@ Return tuple (head, tail) where tail is everything after the final slash. Either part may be empty.""" - seps = _get_bothseps(p) + is_bytes = isinstance(p, bytes) + seps = _get_literal(_bothsep, is_bytes) d, p = splitdrive(p) # set i to index beyond p's last slash i = len(p) @@ -297,8 +297,12 @@ # It is always true that root + ext == p. def splitext(p): - return genericpath._splitext(p, _get_sep(p), _get_altsep(p), - _get_dot(p)) + is_bytes = isinstance(p, bytes) + return genericpath._splitext(p, + _get_literal(_sep, is_bytes), + _get_literal(_altsep, is_bytes), + _get_literal(_dot, is_bytes)) + splitext.__doc__ = genericpath._splitext.__doc__ @@ -344,7 +348,8 @@ def ismount(path): """Test whether a path is a mount point (defined as root of drive)""" - seps = _get_bothseps(path) + is_bytes = isinstance(path, bytes) + seps = _get_literal(_bothsep, is_bytes) root, rest = splitdrive(path) if root and root[0] in seps: return (not rest) or (rest in seps) @@ -364,14 +369,13 @@ """Expand ~ and ~user constructs. If user or $HOME is unknown, do nothing.""" - if isinstance(path, bytes): - tilde = b'~' - else: - tilde = '~' + is_bytes = isinstance(path, bytes) + tilde = _get_literal(_tilde, is_bytes) + if not path.startswith(tilde): return path i, n = 1, len(path) - while i < n and path[i] not in _get_bothseps(path): + while i < n and path[i] not in _get_literal(_bothsep, is_bytes): i += 1 if 'HOME' in os.environ: @@ -387,7 +391,7 @@ drive = '' userhome = join(drive, os.environ['HOMEPATH']) - if isinstance(path, bytes): + if is_bytes: userhome = userhome.encode(sys.getfilesystemencoding()) if i != 1: #~user @@ -413,7 +417,9 @@ """Expand shell variables of the forms $var, ${var} and %var%. Unknown variables are left unchanged.""" - if isinstance(path, bytes): + is_bytes = isinstance(path, bytes) + + if is_bytes: if ord('$') not in path and ord('%') not in path: return path import string @@ -459,13 +465,13 @@ index = pathlen - 1 else: var = path[:index] - if isinstance(path, bytes): + if is_bytes: var = var.decode('ascii') if var in os.environ: value = os.environ[var] else: value = '%' + var + '%' - if isinstance(path, bytes): + if is_bytes: value = value.encode('ascii') res += value elif c == dollar: # variable or '$$' @@ -476,22 +482,22 @@ path = path[index+2:] pathlen = len(path) try: - if isinstance(path, bytes): + if is_bytes: index = path.index(b'}') else: index = path.index('}') var = path[:index] - if isinstance(path, bytes): + if is_bytes: var = var.decode('ascii') if var in os.environ: value = os.environ[var] else: value = '${' + var + '}' - if isinstance(path, bytes): + if is_bytes: value = value.encode('ascii') res += value except ValueError: - if isinstance(path, bytes): + if is_bytes: res += b'${' + path else: res += '${' + path @@ -501,7 +507,7 @@ index += 1 c = path[index:index + 1] while c and c in varchars: - if isinstance(path, bytes): + if is_bytes: var += c.decode('ascii') else: var += c @@ -511,7 +517,7 @@ value = os.environ[var] else: value = '$' + var - if isinstance(path, bytes): + if is_bytes: value = value.encode('ascii') res += value if c: @@ -528,16 +534,18 @@ def normpath(path): """Normalize path, eliminating double slashes, etc.""" - sep = _get_sep(path) - dotdot = _get_dot(path) * 2 - special_prefixes = _get_special(path) + is_bytes = isinstance(path, bytes) + sep = _get_literal(_sep, is_bytes) + dot = _get_literal(_dot, is_bytes) + dotdot = dot * 2 + special_prefixes = _get_special(path, is_bytes) if path.startswith(special_prefixes): # in the case of paths with these prefixes: # \\.\ -> device names # \\?\ -> literal paths # do not do any normalization, but return the path unchanged return path - path = path.replace(_get_altsep(path), sep) + path = path.replace(_get_literal(_altsep, is_bytes), sep) prefix, path = splitdrive(path) # collapse initial backslashes @@ -548,13 +556,13 @@ comps = path.split(sep) i = 0 while i < len(comps): - if not comps[i] or comps[i] == _get_dot(path): + if not comps[i] or comps[i] == dot: del comps[i] elif comps[i] == dotdot: if i > 0 and comps[i-1] != dotdot: del comps[i-1:i+1] i -= 1 - elif i == 0 and prefix.endswith(_get_sep(path)): + elif i == 0 and prefix.endswith(sep): del comps[i] else: i += 1 @@ -562,7 +570,7 @@ i += 1 # If the path is now empty, substitute '.' if not prefix and not comps: - comps.append(_get_dot(path)) + comps.append(dot) return prefix + sep.join(comps) @@ -604,10 +612,11 @@ def relpath(path, start=curdir): """Return a relative version of a path""" - sep = _get_sep(path) + is_bytes = isinstance(path, bytes) + sep = _get_literal(_sep, is_bytes) if start is curdir: - start = _get_dot(path) + start = _get_literal(_dot, is_bytes) if not path: raise ValueError("no path specified") @@ -630,13 +639,11 @@ break i += 1 - if isinstance(path, bytes): - pardir = b'..' - else: - pardir = '..' + dot = _get_literal(_dot, is_bytes) + pardir = dot * 2 rel_list = [pardir] * (len(start_list)-i) + path_list[i:] if not rel_list: - return _get_dot(path) + return dot return join(*rel_list) diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/test/test_ipaddress.py --- a/Lib/test/test_ipaddress.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/test/test_ipaddress.py Sat Jul 07 15:32:29 2012 +0200 @@ -5,415 +5,8 @@ import unittest -import re -import contextlib import ipaddress -class ErrorReporting(unittest.TestCase): - # One big change in ipaddress over the original ipaddr module is - # error reporting that tries to assume users *don't know the rules* - # for what constitutes an RFC compliant IP address - - # Note that if the constructors are refactored so that addresses with - # multiple problems get classified differently, that's OK - just - # move the affected examples to the newly appropriate test case. - - # The error reporting tests also cover a few corner cases that - # specifically *aren't* errors (such as leading zeroes in v6 - # address parts) that don't have an obvious home in the main test - # suite - - @contextlib.contextmanager - def assertCleanError(self, exc_type, details, *args): - """ - Ensure exception does not display a context by default - - Wraps unittest.TestCase.assertRaisesRegex - """ - if args: - details = details % args - cm = self.assertRaisesRegex(exc_type, details) - with cm as exc: - yield exc - # Ensure we produce clean tracebacks on failure - if exc.exception.__context__ is not None: - self.assertTrue(exc.exception.__suppress_context__) - - def assertAddressError(self, details, *args): - """Ensure a clean AddressValueError""" - return self.assertCleanError(ipaddress.AddressValueError, - details, *args) - - def assertNetmaskError(self, details, *args): - """Ensure a clean NetmaskValueError""" - return self.assertCleanError(ipaddress.NetmaskValueError, - details, *args) - -class AddressErrors_v4(ErrorReporting): - - def test_empty_address(self): - with self.assertAddressError("Address cannot be empty"): - ipaddress.IPv4Address("") - - def test_network_passed_as_address(self): - addr = "127.0.0.1/24" - with self.assertAddressError("Unexpected '/' in %r", addr): - ipaddress.IPv4Address(addr) - - def test_bad_address_split(self): - def assertBadSplit(addr): - with self.assertAddressError("Expected 4 octets in %r", addr): - ipaddress.IPv4Address(addr) - - assertBadSplit("127.0.1") - assertBadSplit("42.42.42.42.42") - assertBadSplit("42.42.42") - assertBadSplit("42.42") - assertBadSplit("42") - assertBadSplit("42..42.42.42") - assertBadSplit("42.42.42.42.") - assertBadSplit("42.42.42.42...") - assertBadSplit(".42.42.42.42") - assertBadSplit("...42.42.42.42") - assertBadSplit("016.016.016") - assertBadSplit("016.016") - assertBadSplit("016") - assertBadSplit("000") - assertBadSplit("0x0a.0x0a.0x0a") - assertBadSplit("0x0a.0x0a") - assertBadSplit("0x0a") - assertBadSplit(".") - assertBadSplit("bogus") - assertBadSplit("bogus.com") - assertBadSplit("1000") - assertBadSplit("1000000000000000") - assertBadSplit("192.168.0.1.com") - - def test_empty_octet(self): - def assertBadOctet(addr): - with self.assertAddressError("Empty octet not permitted in %r", - addr): - ipaddress.IPv4Address(addr) - - assertBadOctet("42..42.42") - assertBadOctet("...") - - def test_invalid_characters(self): - def assertBadOctet(addr, octet): - msg = "Only decimal digits permitted in %r in %r" % (octet, addr) - with self.assertAddressError(re.escape(msg)): - ipaddress.IPv4Address(addr) - - assertBadOctet("0x0a.0x0a.0x0a.0x0a", "0x0a") - assertBadOctet("42.42.42.-0", "-0") - assertBadOctet("42.42.42.+0", "+0") - assertBadOctet("42.42.42.-42", "-42") - assertBadOctet("+1.+2.+3.4", "+1") - assertBadOctet("1.2.3.4e0", "4e0") - assertBadOctet("1.2.3.4::", "4::") - assertBadOctet("1.a.2.3", "a") - - def test_leading_zeros(self): - def assertBadOctet(addr, octet): - msg = "Ambiguous leading zero in %r not permitted in %r" - with self.assertAddressError(msg, octet, addr): - ipaddress.IPv4Address(addr) - - assertBadOctet("016.016.016.016", "016") - assertBadOctet("001.000.008.016", "008") - self.assertEqual(ipaddress.IPv4Address("192.168.000.001"), - ipaddress.IPv4Address("192.168.0.1")) - - def test_octet_limit(self): - def assertBadOctet(addr, octet): - msg = "Octet %d (> 255) not permitted in %r" % (octet, addr) - with self.assertAddressError(re.escape(msg)): - ipaddress.IPv4Address(addr) - - assertBadOctet("12345.67899.-54321.-98765", 12345) - assertBadOctet("257.0.0.0", 257) - - def test_bad_packed_length(self): - def assertBadLength(length): - addr = b'\x00' * length - msg = "Packed address %r must be exactly 4 bytes" % addr - with self.assertAddressError(re.escape(msg)): - ipaddress.IPv4Address(addr) - - assertBadLength(3) - assertBadLength(5) - - -class AddressErrors_v6(ErrorReporting): - - def test_empty_address(self): - with self.assertAddressError("Address cannot be empty"): - ipaddress.IPv6Address("") - - def test_network_passed_as_address(self): - addr = "::1/24" - with self.assertAddressError("Unexpected '/' in %r", addr): - ipaddress.IPv6Address(addr) - - def test_bad_address_split_v6_not_enough_parts(self): - def assertBadSplit(addr): - msg = "At least 3 parts expected in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit(":") - assertBadSplit(":1") - assertBadSplit("FEDC:9878") - - def test_bad_address_split_v6_too_many_colons(self): - def assertBadSplit(addr): - msg = "At most 8 colons permitted in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit("9:8:7:6:5:4:3::2:1") - assertBadSplit("10:9:8:7:6:5:4:3:2:1") - assertBadSplit("::8:7:6:5:4:3:2:1") - assertBadSplit("8:7:6:5:4:3:2:1::") - # A trailing IPv4 address is two parts - assertBadSplit("10:9:8:7:6:5:4:3:42.42.42.42") - - def test_bad_address_split_v6_too_many_parts(self): - def assertBadSplit(addr): - msg = "Exactly 8 parts expected without '::' in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit("3ffe:0:0:0:0:0:0:0:1") - assertBadSplit("9:8:7:6:5:4:3:2:1") - assertBadSplit("7:6:5:4:3:2:1") - # A trailing IPv4 address is two parts - assertBadSplit("9:8:7:6:5:4:3:42.42.42.42") - assertBadSplit("7:6:5:4:3:42.42.42.42") - - def test_bad_address_split_v6_too_many_parts_with_double_colon(self): - def assertBadSplit(addr): - msg = "Expected at most 7 other parts with '::' in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit("1:2:3:4::5:6:7:8") - - def test_bad_address_split_v6_repeated_double_colon(self): - def assertBadSplit(addr): - msg = "At most one '::' permitted in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit("3ffe::1::1") - assertBadSplit("1::2::3::4:5") - assertBadSplit("2001::db:::1") - assertBadSplit("3ffe::1::") - assertBadSplit("::3ffe::1") - assertBadSplit(":3ffe::1::1") - assertBadSplit("3ffe::1::1:") - assertBadSplit(":3ffe::1::1:") - assertBadSplit(":::") - assertBadSplit('2001:db8:::1') - - def test_bad_address_split_v6_leading_colon(self): - def assertBadSplit(addr): - msg = "Leading ':' only permitted as part of '::' in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit(":2001:db8::1") - assertBadSplit(":1:2:3:4:5:6:7") - assertBadSplit(":1:2:3:4:5:6:") - assertBadSplit(":6:5:4:3:2:1::") - - def test_bad_address_split_v6_trailing_colon(self): - def assertBadSplit(addr): - msg = "Trailing ':' only permitted as part of '::' in %r" - with self.assertAddressError(msg, addr): - ipaddress.IPv6Address(addr) - - assertBadSplit("2001:db8::1:") - assertBadSplit("1:2:3:4:5:6:7:") - assertBadSplit("::1.2.3.4:") - assertBadSplit("::7:6:5:4:3:2:") - - def test_bad_v4_part_in(self): - def assertBadAddressPart(addr, v4_error): - with self.assertAddressError("%s in %r", v4_error, addr): - ipaddress.IPv6Address(addr) - - assertBadAddressPart("3ffe::1.net", "Expected 4 octets in '1.net'") - assertBadAddressPart("3ffe::127.0.1", - "Expected 4 octets in '127.0.1'") - assertBadAddressPart("::1.2.3", - "Expected 4 octets in '1.2.3'") - assertBadAddressPart("::1.2.3.4.5", - "Expected 4 octets in '1.2.3.4.5'") - assertBadAddressPart("3ffe::1.1.1.net", - "Only decimal digits permitted in 'net' " - "in '1.1.1.net'") - - def test_invalid_characters(self): - def assertBadPart(addr, part): - msg = "Only hex digits permitted in %r in %r" % (part, addr) - with self.assertAddressError(re.escape(msg)): - ipaddress.IPv6Address(addr) - - assertBadPart("3ffe::goog", "goog") - assertBadPart("3ffe::-0", "-0") - assertBadPart("3ffe::+0", "+0") - assertBadPart("3ffe::-1", "-1") - assertBadPart("1.2.3.4::", "1.2.3.4") - assertBadPart('1234:axy::b', "axy") - - def test_part_length(self): - def assertBadPart(addr, part): - msg = "At most 4 characters permitted in %r in %r" - with self.assertAddressError(msg, part, addr): - ipaddress.IPv6Address(addr) - - assertBadPart("3ffe::10000", "10000") - assertBadPart("02001:db8::", "02001") - assertBadPart('2001:888888::1', "888888") - - def test_bad_packed_length(self): - def assertBadLength(length): - addr = b'\x00' * length - msg = "Packed address %r must be exactly 16 bytes" % addr - with self.assertAddressError(re.escape(msg)): - ipaddress.IPv6Address(addr) - - assertBadLength(15) - assertBadLength(17) - - -class NetmaskErrorsMixin_v4: - """Input validation on interfaces and networks is very similar""" - - @property - def factory(self): - raise NotImplementedError - - def test_split_netmask(self): - addr = "1.2.3.4/32/24" - with self.assertAddressError("Only one '/' permitted in %r" % addr): - self.factory(addr) - - def test_address_errors(self): - def assertBadAddress(addr, details): - with self.assertAddressError(details): - self.factory(addr) - - assertBadAddress("", "Address cannot be empty") - assertBadAddress("bogus", "Expected 4 octets") - assertBadAddress("google.com", "Expected 4 octets") - assertBadAddress("10/8", "Expected 4 octets") - assertBadAddress("::1.2.3.4", "Only decimal digits") - assertBadAddress("1.2.3.256", re.escape("256 (> 255)")) - - def test_netmask_errors(self): - def assertBadNetmask(addr, netmask): - msg = "%r is not a valid netmask" - with self.assertNetmaskError(msg % netmask): - self.factory("%s/%s" % (addr, netmask)) - - assertBadNetmask("1.2.3.4", "") - assertBadNetmask("1.2.3.4", "33") - assertBadNetmask("1.2.3.4", "254.254.255.256") - assertBadNetmask("1.1.1.1", "240.255.0.0") - assertBadNetmask("1.1.1.1", "1.a.2.3") - assertBadNetmask("1.1.1.1", "pudding") - - def test_bad_packed_length(self): - def assertBadLength(length): - addr = b'\x00' * length - msg = "Packed address %r must be exactly 4 bytes" % addr - with self.assertAddressError(re.escape(msg)): - self.factory(addr) - - assertBadLength(3) - assertBadLength(5) - - -class InterfaceErrors_v4(ErrorReporting, NetmaskErrorsMixin_v4): - factory = ipaddress.IPv4Interface - -class NetworkErrors_v4(ErrorReporting, NetmaskErrorsMixin_v4): - factory = ipaddress.IPv4Network - - -class NetmaskErrorsMixin_v6: - """Input validation on interfaces and networks is very similar""" - - @property - def factory(self): - raise NotImplementedError - - def test_split_netmask(self): - addr = "cafe:cafe::/128/190" - with self.assertAddressError("Only one '/' permitted in %r" % addr): - self.factory(addr) - - def test_address_errors(self): - def assertBadAddress(addr, details): - with self.assertAddressError(details): - self.factory(addr) - - assertBadAddress("", "Address cannot be empty") - assertBadAddress("google.com", "At least 3 parts") - assertBadAddress("1.2.3.4", "At least 3 parts") - assertBadAddress("10/8", "At least 3 parts") - assertBadAddress("1234:axy::b", "Only hex digits") - - def test_netmask_errors(self): - def assertBadNetmask(addr, netmask): - msg = "%r is not a valid netmask" - with self.assertNetmaskError(msg % netmask): - self.factory("%s/%s" % (addr, netmask)) - - assertBadNetmask("::1", "") - assertBadNetmask("::1", "::1") - assertBadNetmask("::1", "1::") - assertBadNetmask("::1", "129") - assertBadNetmask("::1", "pudding") - - def test_bad_packed_length(self): - def assertBadLength(length): - addr = b'\x00' * length - msg = "Packed address %r must be exactly 16 bytes" % addr - with self.assertAddressError(re.escape(msg)): - self.factory(addr) - - assertBadLength(15) - assertBadLength(17) - - -class InterfaceErrors_v6(ErrorReporting, NetmaskErrorsMixin_v6): - factory = ipaddress.IPv6Interface - -class NetworkErrors_v6(ErrorReporting, NetmaskErrorsMixin_v6): - factory = ipaddress.IPv6Network - - -class FactoryFunctionErrors(ErrorReporting): - - def assertFactoryError(self, factory, kind): - """Ensure a clean ValueError with the expected message""" - addr = "camelot" - msg = '%r does not appear to be an IPv4 or IPv6 %s' - with self.assertCleanError(ValueError, msg, addr, kind): - factory(addr) - - def test_ip_address(self): - self.assertFactoryError(ipaddress.ip_address, "address") - - def test_ip_interface(self): - self.assertFactoryError(ipaddress.ip_interface, "interface") - - def test_ip_network(self): - self.assertFactoryError(ipaddress.ip_network, "network") - class IpaddrUnitTest(unittest.TestCase): @@ -453,6 +46,104 @@ self.assertRaises(ValueError, ipaddress.v6_int_to_packed, 2 ** ipaddress.IPV6LENGTH) + def testInvalidStringsInAddressFactory(self): + def AssertInvalidIP(ip_str): + with self.assertRaises(ValueError) as ex: + ipaddress.ip_address(ip_str) + self.assertIsNone(ex.exception.__context__) + + AssertInvalidIP("") + AssertInvalidIP("016.016.016.016") + AssertInvalidIP("016.016.016") + AssertInvalidIP("016.016") + AssertInvalidIP("016") + AssertInvalidIP("000.000.000.000") + AssertInvalidIP("000") + AssertInvalidIP("0x0a.0x0a.0x0a.0x0a") + AssertInvalidIP("0x0a.0x0a.0x0a") + AssertInvalidIP("0x0a.0x0a") + AssertInvalidIP("0x0a") + AssertInvalidIP("42.42.42.42.42") + AssertInvalidIP("42.42.42") + AssertInvalidIP("42.42") + AssertInvalidIP("42") + AssertInvalidIP("42..42.42") + AssertInvalidIP("42..42.42.42") + AssertInvalidIP("42.42.42.42.") + AssertInvalidIP("42.42.42.42...") + AssertInvalidIP(".42.42.42.42") + AssertInvalidIP("...42.42.42.42") + AssertInvalidIP("42.42.42.-0") + AssertInvalidIP("42.42.42.+0") + AssertInvalidIP(".") + AssertInvalidIP("...") + AssertInvalidIP("bogus") + AssertInvalidIP("bogus.com") + AssertInvalidIP("192.168.0.1.com") + AssertInvalidIP("12345.67899.-54321.-98765") + AssertInvalidIP("257.0.0.0") + AssertInvalidIP("42.42.42.-42") + AssertInvalidIP("3ffe::1.net") + AssertInvalidIP("3ffe::1::1") + AssertInvalidIP("1::2::3::4:5") + AssertInvalidIP("::7:6:5:4:3:2:") + AssertInvalidIP(":6:5:4:3:2:1::") + AssertInvalidIP("2001::db:::1") + AssertInvalidIP("FEDC:9878") + AssertInvalidIP("+1.+2.+3.4") + AssertInvalidIP("1.2.3.4e0") + AssertInvalidIP("::7:6:5:4:3:2:1:0") + AssertInvalidIP("7:6:5:4:3:2:1:0::") + AssertInvalidIP("9:8:7:6:5:4:3::2:1") + AssertInvalidIP("0:1:2:3::4:5:6:7") + AssertInvalidIP("3ffe:0:0:0:0:0:0:0:1") + AssertInvalidIP("3ffe::10000") + AssertInvalidIP("3ffe::goog") + AssertInvalidIP("3ffe::-0") + AssertInvalidIP("3ffe::+0") + AssertInvalidIP("3ffe::-1") + AssertInvalidIP(":") + AssertInvalidIP(":::") + AssertInvalidIP("::1.2.3") + AssertInvalidIP("::1.2.3.4.5") + AssertInvalidIP("::1.2.3.4:") + AssertInvalidIP("1.2.3.4::") + AssertInvalidIP("2001:db8::1:") + AssertInvalidIP(":2001:db8::1") + AssertInvalidIP(":1:2:3:4:5:6:7") + AssertInvalidIP("1:2:3:4:5:6:7:") + AssertInvalidIP(":1:2:3:4:5:6:") + AssertInvalidIP("1000") + AssertInvalidIP("1000000000000000") + AssertInvalidIP("02001:db8::") + self.assertRaises(ValueError, ipaddress.ip_interface, 'bogus') + + def testInvalidStringsInConstructors(self): + def AssertInvalidIP(ip_class, ip_str): + with self.assertRaises(ipaddress.AddressValueError) as ex: + ip_class(ip_str) + if ex.exception.__context__ is not None: + # Provide clean tracebacks by default + self.assertTrue(ex.exception.__suppress_context__) + + AssertInvalidIP(ipaddress.IPv4Address, '127.0.0.1/32') + AssertInvalidIP(ipaddress.IPv4Address(1)._ip_int_from_string, + '1.a.2.3') + AssertInvalidIP(ipaddress.IPv4Interface, '') + AssertInvalidIP(ipaddress.IPv4Interface, 'google.com') + AssertInvalidIP(ipaddress.IPv6Address, '1234:axy::b') + AssertInvalidIP(ipaddress.IPv6Address, '2001:db8:::1') + AssertInvalidIP(ipaddress.IPv6Address, '2001:888888::1') + AssertInvalidIP(ipaddress.IPv4Interface, '::1.2.3.4') + AssertInvalidIP(ipaddress.IPv6Interface, '') + AssertInvalidIP(ipaddress.IPv6Interface, 'google.com') + AssertInvalidIP(ipaddress.IPv6Interface, '1.2.3.4') + AssertInvalidIP(ipaddress.IPv6Interface, 'cafe:cafe::/128/190') + AssertInvalidIP(ipaddress.IPv6Interface, '1234:axy::b') + + def testInvalidHostmask(self): + self.assertFalse(ipaddress.IPv4Interface(1)._is_hostmask('1.a.2.3')) + def testInternals(self): first, last = ipaddress._find_address_range([ ipaddress.IPv4Address('10.10.10.10'), @@ -462,6 +153,7 @@ self.assertEqual(128, ipaddress._count_righthand_zero_bits(0, 128)) self.assertEqual("IPv4Network('1.2.3.0/24')", repr(self.ipv4_network)) self.assertEqual('0x1020318', hex(self.ipv4_network)) + self.assertRaises(TypeError, self.ipv4_network.__eq__, object()) def testMissingAddressVersion(self): class Broken(ipaddress._BaseAddress): @@ -495,22 +187,6 @@ self.assertEqual(str(self.ipv6_network.hostmask), '::ffff:ffff:ffff:ffff') - def testEqualityChecks(self): - # __eq__ should never raise TypeError directly - other = object() - def assertEqualityNotImplemented(instance): - self.assertEqual(instance.__eq__(other), NotImplemented) - self.assertEqual(instance.__ne__(other), NotImplemented) - self.assertFalse(instance == other) - self.assertTrue(instance != other) - - assertEqualityNotImplemented(self.ipv4_address) - assertEqualityNotImplemented(self.ipv4_network) - assertEqualityNotImplemented(self.ipv4_interface) - assertEqualityNotImplemented(self.ipv6_address) - assertEqualityNotImplemented(self.ipv6_network) - assertEqualityNotImplemented(self.ipv6_interface) - def testBadVersionComparison(self): # These should always raise TypeError v4addr = ipaddress.ip_address('1.1.1.1') @@ -602,6 +278,22 @@ self.assertEqual(address('::'), address(b'\x00' * 16)) + def testIpFromPackedErrors(self): + def assertInvalidPackedAddress(f, length): + self.assertRaises(ValueError, f, b'\x00' * length) + assertInvalidPackedAddress(ipaddress.ip_address, 3) + assertInvalidPackedAddress(ipaddress.ip_address, 5) + assertInvalidPackedAddress(ipaddress.ip_address, 15) + assertInvalidPackedAddress(ipaddress.ip_address, 17) + assertInvalidPackedAddress(ipaddress.ip_interface, 3) + assertInvalidPackedAddress(ipaddress.ip_interface, 5) + assertInvalidPackedAddress(ipaddress.ip_interface, 15) + assertInvalidPackedAddress(ipaddress.ip_interface, 17) + assertInvalidPackedAddress(ipaddress.ip_network, 3) + assertInvalidPackedAddress(ipaddress.ip_network, 5) + assertInvalidPackedAddress(ipaddress.ip_network, 15) + assertInvalidPackedAddress(ipaddress.ip_network, 17) + def testGetIp(self): self.assertEqual(int(self.ipv4_interface.ip), 16909060) self.assertEqual(str(self.ipv4_interface.ip), '1.2.3.4') @@ -816,6 +508,38 @@ self.assertFalse(ipaddress.IPv4Network('1.1.0.0/16').__contains__( ipaddress.IPv4Network('1.0.0.0/15'))) + def testBadAddress(self): + self.assertRaises(ipaddress.AddressValueError, ipaddress.IPv4Interface, + 'poop') + self.assertRaises(ipaddress.AddressValueError, + ipaddress.IPv4Interface, '1.2.3.256') + + self.assertRaises(ipaddress.AddressValueError, ipaddress.IPv6Interface, + 'poopv6') + self.assertRaises(ipaddress.AddressValueError, + ipaddress.IPv4Interface, '1.2.3.4/32/24') + self.assertRaises(ipaddress.AddressValueError, + ipaddress.IPv4Network, '1.2.3.4/32/24') + self.assertRaises(ipaddress.AddressValueError, + ipaddress.IPv4Interface, '10/8') + self.assertRaises(ipaddress.AddressValueError, + ipaddress.IPv6Interface, '10/8') + + + def testBadNetMask(self): + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv4Interface, '1.2.3.4/') + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv4Interface, '1.2.3.4/33') + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv4Interface, '1.2.3.4/254.254.255.256') + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv4Interface, '1.1.1.1/240.255.0.0') + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv6Interface, '::1/') + self.assertRaises(ipaddress.NetmaskValueError, + ipaddress.IPv6Interface, '::1/129') + def testNth(self): self.assertEqual(str(self.ipv4_network[5]), '1.2.3.5') self.assertRaises(IndexError, self.ipv4_network.__getitem__, 256) diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/test/test_posix.py --- a/Lib/test/test_posix.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/test/test_posix.py Sat Jul 07 15:32:29 2012 +0200 @@ -1017,6 +1017,9 @@ @unittest.skipUnless(hasattr(os, 'SEEK_HOLE'), "test needs an OS that reports file holes") + @unittest.skipIf(sys.platform in ('freebsd7', 'freebsd8', 'freebsd9'), + "Skip test because known kernel bug - " \ + "http://lists.freebsd.org/pipermail/freebsd-amd64/2012-January/014332.html") def test_fs_holes(self): # Even if the filesystem doesn't report holes, # if the OS supports it the SEEK_* constants @@ -1029,18 +1032,11 @@ fp.flush() size = fp.tell() fno = fp.fileno() - try : - for i in range(size): - self.assertEqual(i, os.lseek(fno, i, os.SEEK_DATA)) - self.assertLessEqual(size, os.lseek(fno, i, os.SEEK_HOLE)) - self.assertRaises(OSError, os.lseek, fno, size, os.SEEK_DATA) - self.assertRaises(OSError, os.lseek, fno, size, os.SEEK_HOLE) - except OSError : - # Some OSs claim to support SEEK_HOLE/SEEK_DATA - # but it is not true. - # For instance: - # http://lists.freebsd.org/pipermail/freebsd-amd64/2012-January/014332.html - raise unittest.SkipTest("OSError raised!") + for i in range(size): + self.assertEqual(i, os.lseek(fno, i, os.SEEK_DATA)) + self.assertLessEqual(size, os.lseek(fno, i, os.SEEK_HOLE)) + self.assertRaises(OSError, os.lseek, fno, size, os.SEEK_DATA) + self.assertRaises(OSError, os.lseek, fno, size, os.SEEK_HOLE) class PosixGroupsTester(unittest.TestCase): diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/test/test_tokenize.py --- a/Lib/test/test_tokenize.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/test/test_tokenize.py Sat Jul 07 15:32:29 2012 +0200 @@ -745,10 +745,6 @@ f = 'tokenize_tests-utf8-coding-cookie-and-utf8-bom-sig.txt' self.assertTrue(self._testFile(f)) - def test_bad_coding_cookie(self): - self.assertRaises(SyntaxError, self._testFile, 'bad_coding.py') - self.assertRaises(SyntaxError, self._testFile, 'bad_coding2.py') - class Test_Tokenize(TestCase): diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/tokenize.py --- a/Lib/tokenize.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/tokenize.py Sat Jul 07 15:32:29 2012 +0200 @@ -388,7 +388,7 @@ raise SyntaxError(msg) if bom_found: - if encoding != 'utf-8': + if codec.name != 'utf-8': # This behaviour mimics the Python interpreter if filename is None: msg = 'encoding problem: utf-8' diff -r 2e9cba1d1554 -r f5c57ba1124b Lib/xml/etree/ElementTree.py --- a/Lib/xml/etree/ElementTree.py Sat Jul 07 23:05:59 2012 +1000 +++ b/Lib/xml/etree/ElementTree.py Sat Jul 07 15:32:29 2012 +0200 @@ -1597,13 +1597,13 @@ type = self._doctype[1] if type == "PUBLIC" and n == 4: name, type, pubid, system = self._doctype - if pubid: - pubid = pubid[1:-1] elif type == "SYSTEM" and n == 3: name, type, system = self._doctype pubid = None else: return + if pubid: + pubid = pubid[1:-1] if hasattr(self.target, "doctype"): self.target.doctype(name, pubid, system[1:-1]) elif self.doctype != self._XMLParser__doctype: diff -r 2e9cba1d1554 -r f5c57ba1124b Misc/NEWS --- a/Misc/NEWS Sat Jul 07 23:05:59 2012 +1000 +++ b/Misc/NEWS Sat Jul 07 15:32:29 2012 +0200 @@ -23,16 +23,6 @@ Library ------- -- Issue #14814: ipaddress network objects correctly return NotImplemented - when compared to arbitrary objects instead of raising TypeError - -- Issue #14990: Correctly fail with SyntaxError on invalid encoding - declaration. - -- Issue #14814: ipaddress now provides more informative error messages when - constructing instances directly (changes permitted during beta due to - provisional API status) - - Issue #15247: FileIO now raises an error when given a file descriptor pointing to a directory. diff -r 2e9cba1d1554 -r f5c57ba1124b Objects/stringlib/codecs.h --- a/Objects/stringlib/codecs.h Sat Jul 07 23:05:59 2012 +1000 +++ b/Objects/stringlib/codecs.h Sat Jul 07 15:32:29 2012 +0200 @@ -8,9 +8,9 @@ /* Mask to quickly check whether a C 'long' contains a non-ASCII, UTF8-encoded char. */ #if (SIZEOF_LONG == 8) -# define ASCII_CHAR_MASK 0x8080808080808080UL +# define ASCII_CHAR_MASK 0x8080808080808080L #elif (SIZEOF_LONG == 4) -# define ASCII_CHAR_MASK 0x80808080UL +# define ASCII_CHAR_MASK 0x80808080L #else # error C 'long' size should be either 4 or 8! #endif diff -r 2e9cba1d1554 -r f5c57ba1124b Objects/stringlib/find_max_char.h --- a/Objects/stringlib/find_max_char.h Sat Jul 07 23:05:59 2012 +1000 +++ b/Objects/stringlib/find_max_char.h Sat Jul 07 15:32:29 2012 +0200 @@ -8,9 +8,9 @@ /* Mask to quickly check whether a C 'long' contains a non-ASCII, UTF8-encoded char. */ #if (SIZEOF_LONG == 8) -# define UCS1_ASCII_CHAR_MASK 0x8080808080808080UL +# define UCS1_ASCII_CHAR_MASK 0x8080808080808080L #elif (SIZEOF_LONG == 4) -# define UCS1_ASCII_CHAR_MASK 0x80808080UL +# define UCS1_ASCII_CHAR_MASK 0x80808080L #else # error C 'long' size should be either 4 or 8! #endif diff -r 2e9cba1d1554 -r f5c57ba1124b Objects/unicodeobject.c --- a/Objects/unicodeobject.c Sat Jul 07 23:05:59 2012 +1000 +++ b/Objects/unicodeobject.c Sat Jul 07 15:32:29 2012 +0200 @@ -4639,9 +4639,9 @@ /* Mask to quickly check whether a C 'long' contains a non-ASCII, UTF8-encoded char. */ #if (SIZEOF_LONG == 8) -# define ASCII_CHAR_MASK 0x8080808080808080UL +# define ASCII_CHAR_MASK 0x8080808080808080L #elif (SIZEOF_LONG == 4) -# define ASCII_CHAR_MASK 0x80808080UL +# define ASCII_CHAR_MASK 0x80808080L #else # error C 'long' size should be either 4 or 8! #endif