msg298474 - (view) |
Author: Cecile Tonglet (cecton) |
Date: 2017-07-17 10:04 |
The IPv6 detection in asyncio.base_events.create_server only detect if IPv6 is available instead of checking if the interface can actually support it.
I noticed that by using Python in a Docker container (example code to reproduce in attachment):
docker run -it --rm -v /tmp/test_ipv6.py:/src/test_ipv6.py python:3.6 python /src/test_ipv6.py
Will result in:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/src/test_ipv6.py", line 11, in <module>
server = loop.run_until_complete(server_creation)
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
% (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
By default Docker containers have only IPv4 enabled:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
38: eth0@if39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 scope global eth0
valid_lft forever preferred_lft forever
I believe this detection mechanism should rely on the interface requested. I found this on the web for Python 2 that manage to get the info per interface: https://pastebin.com/VEnhF1Ht but it's using an external library.
However if you change the hostname to 127.0.0.1 it works normally.
|
msg298683 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-07-19 17:51 |
Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.
Note: errno 99 is EADDRNOTAVAIL.
Something like this could work (untested):
diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index 33b8f48..413161a 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -1038,6 +1038,11 @@ class BaseEventLoop(events.AbstractEventLoop):
try:
sock.bind(sa)
except OSError as err:
+ if err.errno == errno.EADDRNOTAVAIL:
+ # See bpo-30945
+ sockets.pop()
+ sock.close()
+ continue
raise OSError(err.errno, 'error while attempting '
'to bind on address %r: %s'
% (sa, err.strerror.lower())) from None
|
msg298684 - (view) |
Author: Yury Selivanov (yselivanov) * |
Date: 2017-07-19 17:54 |
> Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.
+1
|
msg298714 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-07-20 11:34 |
Cécile, could you try the following patch? I have no easy way to test here.
|
msg298716 - (view) |
Author: Cecile Tonglet (cecton) |
Date: 2017-07-20 11:57 |
Sure! It seems to work, the process returns an exit code of 0 and I see no traceback but the message is still displayed in the terminal.
(Also I did something weird because your patch applies on branch master and I ran it with Python 3.6... I suppose it shouldn't be a problem)
[0] [11:54:13] /tmp > d run -it --rm -v /tmp:/tmp:ro -v ~/repos/cpython/Lib/asyncio:/usr/local/lib/python3.6/asyncio:ro python:3.6 python /tmp/test_ipv6.py
error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[0] [11:54:19] /tmp > d run -it --rm -v /tmp:/tmp:ro python:3.6 python /tmp/test_ipv6.py
Traceback (most recent call last):
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/test_ipv6.py", line 11, in <module>
server = loop.run_until_complete(server_creation)
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
% (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[1] [11:54:52] /tmp >
|
msg298718 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2017-07-20 12:46 |
Cécile, thank you. The reason the message is still displayed is that I turned the error into a warning (in other words, it is expected).
|
msg311915 - (view) |
Author: Anthony Sottile (Anthony Sottile) * |
Date: 2018-02-09 23:32 |
Seeing this as well when running the cpython test suite in docker:
```
$ ./python -m test.test_asyncio
...
[18 similar traces omitted]
======================================================================
ERROR: test_sock_sendfile_zero_size (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/code/Lib/test/test_asyncio/test_unix_events.py", line 559, in test_sock_sendfile_zero_size
sock, proto = self.prepare()
File "/code/Lib/test/test_asyncio/test_unix_events.py", line 483, in prepare
lambda: proto, support.HOST, port))
File "/code/Lib/test/test_asyncio/test_unix_events.py", line 476, in run_loop
return self.loop.run_until_complete(coro)
File "/code/Lib/asyncio/base_events.py", line 566, in run_until_complete
return future.result()
File "/code/Lib/asyncio/base_events.py", line 1346, in create_server
% (sa, err.strerror.lower())) from None
OSError: [Errno 99] error while attempting to bind on address ('::1', 39527, 0, 0): cannot assign requested address
----------------------------------------------------------------------
```
I'm going to try and write a patch to skip these tests (there's already a helper)
|
msg311916 - (view) |
Author: Anthony Sottile (Anthony Sottile) * |
Date: 2018-02-09 23:52 |
Actually, my issue seems to be something more strange.
The host being passed in is `localhost` which resolves to:
```
>>> pprint.pprint(socket.getaddrinfo('localhost', 80))
[(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_STREAM: 1>,
6,
'',
('127.0.0.1', 80)),
(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_DGRAM: 2>,
17,
'',
('127.0.0.1', 80)),
(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_RAW: 3>,
0,
'',
('127.0.0.1', 80)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_STREAM: 1>,
6,
'',
('::1', 80, 0, 0)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_DGRAM: 2>,
17,
'',
('::1', 80, 0, 0)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_RAW: 3>,
0,
'',
('::1', 80, 0, 0))]
```
asyncio is picking ipv6 because of this code:
https://github.com/python/cpython/blob/a445feb72902e4a3c5ae712f0c289309e1580d52/Lib/asyncio/base_events.py#L1334-L1340
despite my host not actually having an ipv6 network hooked up.
|
msg311917 - (view) |
Author: Anthony Sottile (Anthony Sottile) * |
Date: 2018-02-09 23:53 |
Applying this patch makes the tests pass for me, but I don't think the patch is appropriate (just hides the bug):
```
$ git diff
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
index 5bd76d3..ff6c4e1 100644
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -480,7 +480,7 @@ class SelectorEventLoopUnixSockSendfileTests(test_utils.TestCase):
proto = self.MyProto(self.loop)
port = support.find_unused_port()
server = self.run_loop(self.loop.create_server(
- lambda: proto, support.HOST, port))
+ lambda: proto, support.HOSTv4, port))
self.run_loop(self.loop.sock_connect(sock, (support.HOST, port)))
def cleanup():
```
|
msg311942 - (view) |
Author: Cecile Tonglet (cecton) |
Date: 2018-02-10 07:25 |
I see that the patch hasn't been applied to master on GitHub. Is there anything else expected from me on this ticket?
|
msg317895 - (view) |
Author: Yury Selivanov (yselivanov) * |
Date: 2018-05-28 17:49 |
Does anybody wants to make a PR to fix this?
|
msg318045 - (view) |
Author: Yury Selivanov (yselivanov) * |
Date: 2018-05-29 15:29 |
> asyncio is picking ipv6 because of this code:
> https://github.com/python/cpython/blob/a445feb72902e4a3c5ae712f0c289309e1580d52/Lib/asyncio/base_events.py#L1334-L1340
> despite my host not actually having an ipv6 network hooked up.
Which should be fine; create_server enumerates all addresses and tries to connect to each one. IPV6_V6ONLY is only applied to one socket for one IPv6 address. I think Antoine's patch is OK.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:58:49 | admin | set | github: 75128 |
2018-10-19 21:14:34 | jnwatson | set | nosy:
+ jnwatson
|
2018-05-29 15:40:38 | yselivanov | set | stage: patch review pull_requests:
+ pull_request6842 |
2018-05-29 15:29:12 | yselivanov | set | messages:
+ msg318045 |
2018-05-28 17:49:13 | yselivanov | set | messages:
+ msg317895 |
2018-02-10 07:25:17 | cecton | set | messages:
+ msg311942 |
2018-02-09 23:53:38 | Anthony Sottile | set | messages:
+ msg311917 |
2018-02-09 23:52:27 | Anthony Sottile | set | messages:
+ msg311916 |
2018-02-09 23:32:42 | Anthony Sottile | set | nosy:
+ Anthony Sottile messages:
+ msg311915
|
2017-07-20 12:46:05 | pitrou | set | messages:
+ msg298718 |
2017-07-20 11:57:41 | cecton | set | messages:
+ msg298716 |
2017-07-20 11:34:42 | pitrou | set | files:
+ eaddrnotavail_asyncio.patch keywords:
+ patch messages:
+ msg298714
|
2017-07-19 17:54:25 | yselivanov | set | messages:
+ msg298684 |
2017-07-19 17:51:14 | pitrou | set | nosy:
+ pitrou
messages:
+ msg298683 versions:
+ Python 3.7, - Python 3.4 |
2017-07-17 10:04:34 | cecton | create | |