classification
Title: loop.create_server does not detect if the interface is IPv6 enabled
Type: behavior Stage:
Components: asyncio Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, cecton, pitrou, yselivanov
Priority: normal Keywords: patch

Created on 2017-07-17 10:04 by cecton, last changed 2018-02-10 07:25 by cecton.

Files
File name Uploaded Description Edit
test_ipv6.py cecton, 2017-07-17 10:04 Example file to reproduce the issue
eaddrnotavail_asyncio.patch pitrou, 2017-07-20 11:34
Messages (10)
msg298474 - (view) Author: Cecile Tonglet (cecton) Date: 2017-07-17 10:04
The IPv6 detection in asyncio.base_events.create_server only detect if IPv6 is available instead of checking if the interface can actually support it.

I noticed that by using Python in a Docker container (example code to reproduce in attachment):


docker run -it --rm -v /tmp/test_ipv6.py:/src/test_ipv6.py python:3.6 python /src/test_ipv6.py


Will result in:


Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/src/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address


By default Docker containers have only IPv4 enabled:


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
38: eth0@if39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.2/16 scope global eth0
       valid_lft forever preferred_lft forever


I believe this detection mechanism should rely on the interface requested. I found this on the web for Python 2 that manage to get the info per interface: https://pastebin.com/VEnhF1Ht but it's using an external library.

However if you change the hostname to 127.0.0.1 it works normally.
msg298683 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-07-19 17:51
Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.

Note: errno 99 is EADDRNOTAVAIL.

Something like this could work (untested):

diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index 33b8f48..413161a 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -1038,6 +1038,11 @@ class BaseEventLoop(events.AbstractEventLoop):
                     try:
                         sock.bind(sa)
                     except OSError as err:
+                        if err.errno == errno.EADDRNOTAVAIL:
+                            # See bpo-30945
+                            sockets.pop()
+                            sock.close()
+                            continue
                         raise OSError(err.errno, 'error while attempting '
                                       'to bind on address %r: %s'
                                       % (sa, err.strerror.lower())) from None
msg298684 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2017-07-19 17:54
> Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.


+1
msg298714 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-07-20 11:34
Cécile, could you try the following patch?  I have no easy way to test here.
msg298716 - (view) Author: Cecile Tonglet (cecton) Date: 2017-07-20 11:57
Sure! It seems to work, the process returns an exit code of 0 and I see no traceback but the message is still displayed in the terminal.

(Also I did something weird because your patch applies on branch master and I ran it with Python 3.6... I suppose it shouldn't be a problem)


[0] [11:54:13] /tmp > d run -it --rm -v /tmp:/tmp:ro -v ~/repos/cpython/Lib/asyncio:/usr/local/lib/python3.6/asyncio:ro python:3.6 python /tmp/test_ipv6.py
error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[0] [11:54:19] /tmp > d run -it --rm -v /tmp:/tmp:ro python:3.6 python /tmp/test_ipv6.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[1] [11:54:52] /tmp >
msg298718 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-07-20 12:46
Cécile, thank you.  The reason the message is still displayed is that I turned the error into a warning (in other words, it is expected).
msg311915 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-02-09 23:32
Seeing this as well when running the cpython test suite in docker:

```
$ ./python -m test.test_asyncio

...

[18 similar traces omitted]
======================================================================
ERROR: test_sock_sendfile_zero_size (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 559, in test_sock_sendfile_zero_size
    sock, proto = self.prepare()
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 483, in prepare
    lambda: proto, support.HOST, port))
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 476, in run_loop
    return self.loop.run_until_complete(coro)
  File "/code/Lib/asyncio/base_events.py", line 566, in run_until_complete
    return future.result()
  File "/code/Lib/asyncio/base_events.py", line 1346, in create_server
    % (sa, err.strerror.lower())) from None
OSError: [Errno 99] error while attempting to bind on address ('::1', 39527, 0, 0): cannot assign requested address

----------------------------------------------------------------------

```


I'm going to try and write a patch to skip these tests (there's already a helper)
msg311916 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-02-09 23:52
Actually, my issue seems to be something more strange.

The host being passed in is `localhost` which resolves to:

```
>>> pprint.pprint(socket.getaddrinfo('localhost', 80))
[(<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('::1', 80, 0, 0))]
```

asyncio is picking ipv6 because of this code:

https://github.com/python/cpython/blob/a445feb72902e4a3c5ae712f0c289309e1580d52/Lib/asyncio/base_events.py#L1334-L1340

despite my host not actually having an ipv6 network hooked up.
msg311917 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-02-09 23:53
Applying this patch makes the tests pass for me, but I don't think the patch is appropriate (just hides the bug):

```
$ git diff
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
index 5bd76d3..ff6c4e1 100644
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -480,7 +480,7 @@ class SelectorEventLoopUnixSockSendfileTests(test_utils.TestCase):
         proto = self.MyProto(self.loop)
         port = support.find_unused_port()
         server = self.run_loop(self.loop.create_server(
-            lambda: proto, support.HOST, port))
+            lambda: proto, support.HOSTv4, port))
         self.run_loop(self.loop.sock_connect(sock, (support.HOST, port)))
 
         def cleanup():
```
msg311942 - (view) Author: Cecile Tonglet (cecton) Date: 2018-02-10 07:25
I see that the patch hasn't been applied to master on GitHub. Is there anything else expected from me on this ticket?
History
Date User Action Args
2018-02-10 07:25:17cectonsetmessages: + msg311942
2018-02-09 23:53:38Anthony Sottilesetmessages: + msg311917
2018-02-09 23:52:27Anthony Sottilesetmessages: + msg311916
2018-02-09 23:32:42Anthony Sottilesetnosy: + Anthony Sottile
messages: + msg311915
2017-07-20 12:46:05pitrousetmessages: + msg298718
2017-07-20 11:57:41cectonsetmessages: + msg298716
2017-07-20 11:34:42pitrousetfiles: + eaddrnotavail_asyncio.patch
keywords: + patch
messages: + msg298714
2017-07-19 17:54:25yselivanovsetmessages: + msg298684
2017-07-19 17:51:14pitrousetnosy: + pitrou

messages: + msg298683
versions: + Python 3.7, - Python 3.4
2017-07-17 10:04:34cectoncreate