classification
Title: test_nntplib: random EOFError in setUpClass()
Type: Stage: backport needed
Components: Tests Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: inada.naoki, koobs, pitrou, vstinner
Priority: normal Keywords:

Created on 2017-04-27 15:07 by vstinner, last changed 2017-09-29 15:51 by vstinner.

Files
File name Uploaded Description Edit
koobs-freebsd-current-debug.py36.build.419.log koobs, 2017-08-25 04:49
Pull Requests
URL Status Linked Edit
PR 1322 closed vstinner, 2017-04-27 15:09
PR 1323 merged vstinner, 2017-04-27 15:38
PR 1324 merged vstinner, 2017-04-27 15:40
PR 2843 merged vstinner, 2017-07-24 13:37
PR 2892 merged inada.naoki, 2017-07-26 12:10
Messages (15)
msg292450 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 15:07
Example of failure:

======================================================================
ERROR: setUpClass (test.test_nntplib.NetworkedNNTPTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/haypo/prog/python/master/Lib/test/test_nntplib.py", line 289, in setUpClass
    cls.server = cls.NNTP_CLASS(cls.NNTP_HOST, timeout=TIMEOUT, usenetrc=False)
  File "/home/haypo/prog/python/master/Lib/nntplib.py", line 1048, in __init__
    readermode, timeout)
  File "/home/haypo/prog/python/master/Lib/nntplib.py", line 330, in __init__
    self.welcome = self._getresp()
  File "/home/haypo/prog/python/master/Lib/nntplib.py", line 449, in _getresp
    resp = self._getline()
  File "/home/haypo/prog/python/master/Lib/nntplib.py", line 437, in _getline
    if not line: raise EOFError
EOFError

----------------------------------------------------------------------

Attached PR catch this error and skips the test.

See also issue #19613 and #19756.
msg292453 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 15:30
New changeset 5bccca58b9b2b3a925b16750bedbd907695ea8d7 by Victor Stinner in branch 'master':
bpo-30175: Skip client cert tests of test_imaplib (#1320)
https://github.com/python/cpython/commit/5bccca58b9b2b3a925b16750bedbd907695ea8d7
msg292454 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 15:30
NetworkedNNTPTests uses the NNTP server news.trigofacile.com (port 119). It's possible to connect to this server (IPv4: 91.121.26.68), but recv() returns an empty string.
msg292455 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 15:32
I see 3 options:

* Contact news.trigofacile.com administrator to ask him to repair the NNTP server
* Run a NNTP server on pythontest.net
* Mock the server: see http://bugs.python.org/issue19756#msg268841
msg292456 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 15:35
The news.trigofacile.com server was chosen in the commit 1cb121eceae698b1be0b383de5d42dc9b9accd02 (Antoine Pitrou in 2010) which also added the new NetworkedNNTPTests test case.
msg292463 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 16:25
New changeset 4dc3b9cf2a868a8378ecbc2221a15e7b59e8e944 by Victor Stinner in branch '3.6':
bpo-30175: Skip client cert tests of test_imaplib (#1320) (#1323)
https://github.com/python/cpython/commit/4dc3b9cf2a868a8378ecbc2221a15e7b59e8e944
msg292473 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 19:32
New changeset 0eda2d43a7d5b262c979b944592999015a822395 by Victor Stinner in branch '3.5':
bpo-30175: Skip client cert tests of test_imaplib (#1320) (#1324)
https://github.com/python/cpython/commit/0eda2d43a7d5b262c979b944592999015a822395
msg292488 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-04-27 23:25
> Contact news.trigofacile.com administrator to ask him to repair the NNTP server

I did that and I got a reply: the server is now fixed! :-D

The test is now skipped on server failure (in 3.5, 3.6 and master), and the server is repaired. So I close the issue.

Note: Python 2.7 doesn't have the unit test and so is not impacted.
msg298966 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-24 13:35
Similar but different error (ssl.SSLEOFError) on AMD64 FreeBSD CURRENT Non-Debug 3.5:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%20CURRENT%20Non-Debug%203.5/builds/211/steps/test/logs/stdio

======================================================================
ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/test/test_nntplib.py", line 291, in setUpClass
    usenetrc=False)
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/nntplib.py", line 1077, in __init__
    self.sock = _encrypt_on(self.sock, ssl_context, host)
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:719)
msg298985 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-07-24 15:41
New changeset 5b4feb7e86ecb813b2c56560f86cda2fd46b9579 by Victor Stinner in branch 'master':
bpo-30188: test_nntplib catch also ssl.SSLEOFError (#2843)
https://github.com/python/cpython/commit/5b4feb7e86ecb813b2c56560f86cda2fd46b9579
msg299235 - (view) Author: INADA Naoki (inada.naoki) * (Python committer) Date: 2017-07-26 14:43
New changeset 067931dd95ddfa34297be9c602a796d9f8d9bea3 by INADA Naoki in branch 'master':
bpo-30188: fix TypeError in test_nntplib (GH-2892)
https://github.com/python/cpython/commit/067931dd95ddfa34297be9c602a796d9f8d9bea3
msg300823 - (view) Author: Kubilay Kocak (koobs) Date: 2017-08-25 04:49
bpo-30188 (and bpo-30188 ?) need merging to 3.6/3.5, 3.6 just failed with:

ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:748)

Full build log attached
msg300824 - (view) Author: Kubilay Kocak (koobs) Date: 2017-08-25 04:52
Apologies I meant 5b4feb7e86 by haypo and 067931dd95d (also needs merge?)  by inada.naoki
msg302065 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-13 12:28
The SSL EOF error is still common, but in the test, not in setUpClass(). Example:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%2010.x%20Shared%203.x/builds/874/steps/test/logs/stdio

======================================================================
ERROR: test_with_statement (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_nntplib.py", line 241, in wrapped
    meth(self)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/test/test_nntplib.py", line 263, in test_with_statement
    with self.NNTP_CLASS(self.NNTP_HOST, timeout=TIMEOUT, usenetrc=False) as server:
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/nntplib.py", line 1077, in __init__
    self.sock = _encrypt_on(self.sock, ssl_context, host)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:864)

pythininfo:

ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.0.1u-freebsd  22 Sep 2016
ssl.OPENSSL_VERSION_INFO: (1, 0, 1, 21, 15)
ssl.OP_ALL: 0x800003ff
ssl.OP_NO_TLSv1_1: 0x10000000
msg303334 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-09-29 15:51
Recent failure:

http://buildbot.python.org/all/builders/x86%20Gentoo%20Non-Debug%20with%20X%203.x/builds/1080/steps/test/logs/stdio

======================================================================
ERROR: test_with_statement (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_nntplib.py", line 241, in wrapped
    meth(self)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/test/test_nntplib.py", line 263, in test_with_statement
    with self.NNTP_CLASS(self.NNTP_HOST, timeout=TIMEOUT, usenetrc=False) as server:
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/nntplib.py", line 1077, in __init__
    self.sock = _encrypt_on(self.sock, ssl_context, host)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/ssl.py", line 411, in wrap_socket
    _session=session
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/ssl.py", line 822, in __init__
    self.do_handshake()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/ssl.py", line 1076, in do_handshake
    self._sslobj.do_handshake()
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.nondebug/build/Lib/ssl.py", line 697, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:864)
History
Date User Action Args
2017-09-29 15:51:55vstinnersetmessages: + msg303334
2017-09-13 12:28:27vstinnersetmessages: + msg302065
2017-08-25 04:52:18koobssetmessages: + msg300824
2017-08-25 04:49:55koobssetstatus: closed -> open
files: + koobs-freebsd-current-debug.py36.build.419.log


nosy: + koobs
messages: + msg300823
resolution: fixed ->
stage: resolved -> backport needed
2017-08-11 00:09:45vstinnersetstatus: open -> closed
resolution: fixed
2017-07-26 14:43:27inada.naokisetnosy: + inada.naoki
messages: + msg299235
2017-07-26 12:10:29inada.naokisetpull_requests: + pull_request2944
2017-07-24 15:41:05vstinnersetmessages: + msg298985
2017-07-24 13:37:25vstinnersetpull_requests: + pull_request2887
2017-07-24 13:35:31vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg298966
2017-04-27 23:25:40vstinnersetstatus: open -> closed
versions: + Python 3.5, Python 3.6
messages: + msg292488

resolution: fixed
stage: resolved
2017-04-27 19:32:12vstinnersetmessages: + msg292473
2017-04-27 16:25:06vstinnersetmessages: + msg292463
2017-04-27 15:40:00vstinnersetpull_requests: + pull_request1435
2017-04-27 15:38:50vstinnersetpull_requests: + pull_request1433
2017-04-27 15:35:00vstinnersetnosy: + pitrou
messages: + msg292456
2017-04-27 15:32:20vstinnersetmessages: + msg292455
2017-04-27 15:30:57vstinnersetmessages: + msg292454
2017-04-27 15:30:15vstinnersetmessages: + msg292453
2017-04-27 15:09:37vstinnersetpull_requests: + pull_request1430
2017-04-27 15:07:08vstinnercreate