classification
Title: create_connection with local_addr misses valid socket bindings
Type: behavior Stage: patch review
Components: asyncio Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, kyuupichan, ronaldoussoren, twisteroid ambassador, yselivanov
Priority: normal Keywords: patch

Created on 2018-11-23 18:30 by kyuupichan, last changed 2018-12-20 13:02 by twisteroid ambassador.

Pull Requests
URL Status Linked Edit
PR 11241 open twisteroid ambassador, 2018-12-19 13:34
Messages (4)
msg330354 - (view) Author: Neil Booth (kyuupichan) * Date: 2018-11-23 18:30
I run a machine with IPv4 and IPv6 interfaces on MacOSX Mojave.

I try to loop.create_connection() to a remote machine whose domain resolves to an IPv6 address only, with a local_addr domain name argument that resolves to two local addresses: an IPv4 one first and an IPv6 one second.

The loop https://github.com/python/cpython/blob/master/Lib/asyncio/base_events.py#L927-L943 that loops through the local addresses, IPv4 then IPv6, successfully binds the IPv4 laddr_info to the IPv6 socket successfully (something I find surprising) and then the connection attempt fails with OSError(65, 'No route to host'), at which point the sockets is closed and the IPv6 laddr_info is never tried and the connection attempt fails.

If I reverse the order of the loop so that the IPv6 laddr_info is tried first then the connection succeeds.

I suggest either all laddr_info bindings should be tried for each outer loop of infos, rather than just 1, or that those of a different socket "type" (IPv4 vs IPv6) should be skipped in the inner loop before attempting a binding.
msg332136 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2018-12-19 13:18
IMO macOS is at fault here, for even allowing an IPv6 socket to bind to an IPv4 address. ;-)

I have given some thought about this issue when writing my happy eyeballs library. My current solution is closest to Neil's first suggestion, i.e. each pair of remote addrinfo and local addrinfo is tried in a connection attempt.
msg332147 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2018-12-19 16:12
A better workaround is IMHO to force the socket to be IPV6 only:

sd = socket.socket(socket.AF_INET6, socket.SOCK_STREAM, 0)
sd.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)

That avoids the ordering problem as well as having to try all possible combinations of source and destination addreses.

I've tested that setting this option makes it impossible to bind a IPv6 socket to an IPv4 address.  This is on macOS 10.14.2, I haven't checked other versions of macOS.
msg332224 - (view) Author: twisteroid ambassador (twisteroid ambassador) * Date: 2018-12-20 13:02
I don't have a Mac, so I have not tested Ronald's workaround. Assuming it works, we will have to either i) implement platform-specific behavior and only apply IPV6_V6ONLY on macOS for each AF_INET6 socket created, or ii) apply it to all AF_INET6 sockets on all platforms, ideally after testing the option on all these platforms to make sure it doesn't have any undesirable side effect.

Linux's man page of ipv6 (http://man7.org/linux/man-pages/man7/ipv6.7.html ) has this to say about IPV6_V6ONLY:


If this flag is set to true (nonzero), then the socket is re‐
stricted to sending and receiving IPv6 packets only.  In this
case, an IPv4 and an IPv6 application can bind to a single
port at the same time.

If this flag is set to false (zero), then the socket can be
used to send and receive packets to and from an IPv6 address
or an IPv4-mapped IPv6 address.


So setting IPV6_V6ONLY might break some use cases? I have no idea how prevalent that may be.

The upside of this solution, as well as the second suggestion in Neil's OP (filter out local addrinfos with mismatching family), is that they should not increase connect time for normal cases. My solution (for which I have already submitted a PR) probably has a negligible increase in connection time and resource usage, because a fresh socket object is created for each pair of remote and local addrinfo.
History
Date User Action Args
2018-12-20 13:02:23twisteroid ambassadorsetmessages: + msg332224
2018-12-19 16:12:53ronaldoussorensetnosy: + ronaldoussoren
messages: + msg332147
2018-12-19 13:34:28twisteroid ambassadorsetkeywords: + patch
stage: patch review
pull_requests: + pull_request10471
2018-12-19 13:18:49twisteroid ambassadorsetnosy: + twisteroid ambassador
messages: + msg332136
2018-11-23 18:30:53kyuupichancreate