classification
Title: Random failure of test_TCPServer() of test.test_socketserver.SocketServerTest and test_handle_accept() of test.test_asyncore.TestAPI_UseIPv6Select on FreeBSD buildbots
Type: behavior Stage: needs patch
Components: Tests Versions: Python 3.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Change socket.close() to ignore ECONNRESET
View: 30319
Assigned To: Nosy List: berker.peksag, haypo, koobs, martin.panter
Priority: normal Keywords:

Created on 2016-08-17 08:23 by haypo, last changed 2017-07-04 14:56 by haypo. This issue is now closed.

Messages (5)
msg272910 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-08-17 08:23
http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.x%20Shared%203.x/builds/4760/steps/test/logs/stdio

test_handle_accept (test.test_asyncore.TestAPI_UseIPv6Select) ... ERROR

======================================================================
ERROR: test_handle_accept (test.test_asyncore.TestAPI_UseIPv6Select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_asyncore.py", line 500, in tearDown
    asyncore.close_all()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/asyncore.py", line 561, in close_all
    x.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/asyncore.py", line 397, in close
    self.socket.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socket.py", line 407, in close
    self._real_close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socket.py", line 401, in _real_close
    _ss.close(self)
ConnectionResetError: [Errno 54] Connection reset by peer

----------------------------------------------------------------------


0:15:27 [319/402/4] test_socketserver failed -- running: test_tokenize (190 sec), test_datetime (279 sec), test_tools (194 sec)
(...)
test_TCPServer (test.test_socketserver.SocketServerTest) ... creating server
ADDR = ('127.0.0.1', 42875)
CLASS = <class 'socketserver.TCPServer'>
server running
test client 0
test client 1
Exception in thread <class 'socketserver.TCPServer'> serving:
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 238, in serve_forever
    self._handle_request_noblock()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 319, in _handle_request_noblock
    self.handle_error(request, client_address)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 317, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 349, in process_request
    self.shutdown_request(request)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 510, in shutdown_request
    self.close_request(request)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socketserver.py", line 514, in close_request
    request.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socket.py", line 407, in close
    self._real_close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/socket.py", line 401, in _real_close
    _ss.close(self)
ConnectionResetError: [Errno 54] Connection reset by peer

ERROR
/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/unittest/case.py:628: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 42877), raddr=('127.0.0.1', 42875)>
  outcome.errors.clear()
/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/unittest/case.py:628: ResourceWarning: unclosed <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 42875)>
  outcome.errors.clear()
(...)
======================================================================
ERROR: test_TCPServer (test.test_socketserver.SocketServerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_socketserver.py", line 175, in test_TCPServer
    self.stream_examine)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/support/__init__.py", line 1956, in decorator
    return func(*args)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_socketserver.py", line 141, in run_server
    testfunc(svrcls.address_family, addr)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_socketserver.py", line 153, in stream_examine
    buf = data = receive(s, 100)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_socketserver.py", line 46, in receive
    raise RuntimeError("timed out on %r" % (sock,))
RuntimeError: timed out on <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 42877), raddr=('127.0.0.1', 42875)>

----------------------------------------------------------------------
Ran 25 tests in 25.825s
msg273001 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-08-17 23:52
I think I have seen these kind of errors pop up randomly on various tests on Free BSD buildbots (at least in the last few months or so).

Are the buildbots so overloaded that they drop localhost connections? Or are localhost TCP connections generally just that unreliable on Free BSD? Or is something else going on? I’m not faimilar with the OS.

This only seems to affect 3.6, not the equivalent 3.5 buildbot.
msg273382 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-08-22 15:52
Here is another one on koobs-freebsd10: http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.x%20Shared%203.x/builds/4809/steps/test/logs/stdio test_handle_accept was fine this time.

And this one is from koobs-freebsd9:

======================================================================
ERROR: test_handle_accept (test.test_asyncore.TestAPI_UseIPv6Select)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/test/test_asyncore.py", line 500, in tearDown
    asyncore.close_all()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/asyncore.py", line 561, in close_all
    x.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/asyncore.py", line 397, in close
    self.socket.close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/socket.py", line 407, in close
    self._real_close()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd9/build/Lib/socket.py", line 401, in _real_close
    _ss.close(self)
ConnectionResetError: [Errno 54] Connection reset by peer

----------------------------------------------------------------------

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.x%203.x/builds/4705/steps/test/logs/stdio
msg282444 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-12-05 17:35
I didn't look at buildbots recently, and I don't have access to FreeBSD, so I just close the issue :-/
msg297667 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-07-04 14:56
Duplicate of bpo-30319.
History
Date User Action Args
2017-07-04 14:56:54hayposetmessages: + msg297667
2017-07-04 14:54:42haypounlinkissue30319 dependencies
2017-07-04 14:53:30hayposetsuperseder: Change socket.close() to ignore ECONNRESET
resolution: out of date -> duplicate
2017-06-03 22:26:56martin.panterlinkissue30319 dependencies
2016-12-05 17:35:40hayposetstatus: open -> closed
resolution: out of date
messages: + msg282444
2016-08-22 15:52:30berker.peksagsetnosy: + berker.peksag
messages: + msg273382

components: + Tests
type: behavior
stage: needs patch
2016-08-17 23:52:04martin.pantersetnosy: + martin.panter

messages: + msg273001
versions: + Python 3.6
2016-08-17 08:23:46hayposetnosy: + koobs
2016-08-17 08:23:42haypocreate