Just to be complete, the test also fails on x86 Gentoo Non-Debug with X 3.x:
http://buildbot.python.org/all/#/builders/99/builds/564

...
test_start_server_1 (test.test_asyncio.test_server.SelectorStartServerTests) ... Exception in thread test-client:
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/functional.py", line 193, in run
    self._prog(TestSocketWrapper(self._sock))
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/test_server.py", line 44, in <lambda>
    with self.tcp_client(lambda sock: client(sock, addr)):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/test_server.py", line 19, in client
    sock.connect(addr)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/functional.py", line 195, in run
    self._test._abort_socket_test(ex)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/functional.py", line 122, in _abort_socket_test
    self.fail(ex)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/unittest/case.py", line 680, in fail
    raise self.failureException(msg)
AssertionError: [Errno 111] Connection refused

Timeout (0:15:00)!
Thread 0xb7d69700 (most recent call first):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/selectors.py", line 468 in select
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/asyncio/base_events.py", line 1555 in _run_once
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/asyncio/base_events.py", line 450 in run_forever
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/asyncio/base_events.py", line 482 in run_until_complete
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_asyncio/test_server.py", line 45 in test_start_server_1
...