classification
Title: test_ftplib is failing with TLS 1.3
Type: behavior Stage: resolved
Components: SSL Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: christian.heimes, terry.reedy
Priority: normal Keywords: patch

Created on 2018-08-13 09:50 by christian.heimes, last changed 2021-04-19 20:13 by christian.heimes. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8787 merged christian.heimes, 2018-08-16 16:51
PR 8789 merged miss-islington, 2018-08-16 17:44
PR 8790 merged miss-islington, 2018-08-16 17:44
PR 8791 merged christian.heimes, 2018-08-16 17:46
Messages (8)
msg323472 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-13 09:50
Related to #32947

Four ftplib tests are failing with OpenSSL 1.1.1-pre8 and TLS 1.3 enabled. All failing tests use a separate data connection and transfer data on the server. For store operations, the client never calls recv(). This breaks bidirectional shutdown. I assume there are session tickets stuck on the wire.

======================================================================
ERROR: test_storbinary (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 591, in test_storbinary
    self.client.storbinary('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storbinary_rest (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 603, in test_storbinary_rest
    self.client.storbinary('stor', f, rest=r)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storlines (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 608, in test_storlines
    self.client.storlines('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 545, in storlines
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_data_connection (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 889, in test_data_connection
    self.assertEqual(self.client.voidresp(), "226 transfer complete")
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 251, in voidresp
    resp = self.getresp()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 236, in getresp
    resp = self.getmultiline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 222, in getmultiline
    line = self.getline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 204, in getline
    line = self.file.readline(self.maxline + 1)
  File "/home/heimes/dev/python/cpython/Lib/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

----------------------------------------------------------------------
Ran 88 tests in 9.402s

FAILED (errors=4, skipped=1)
msg323608 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-16 16:40
The "OSError: [Errno 0] Error" failures are caused by session ticket handling in two way shutdown. I reported the issue a while ago https://github.com/openssl/openssl/issues/6262 and it's fixed in OpenSSL git master (to be released as 1.1.1-pre9).

The error in test_data_connection is actually "[SSL] shutdown while in init". The dummy server code is failing, because the client starts a two way shutdown before the full handshake has been established. A simple recv() call is good enough to finalize the handshake.
msg323612 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-16 17:43
New changeset 1590c393360df059160145e7475754427bfc6680 by Christian Heimes in branch 'master':
bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
https://github.com/python/cpython/commit/1590c393360df059160145e7475754427bfc6680
msg323615 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-16 19:37
New changeset 870247a57e84ccaa3f6a6ce955f4168632b967a8 by Christian Heimes (Miss Islington (bot)) in branch '3.7':
[3.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (GH-8789)
https://github.com/python/cpython/commit/870247a57e84ccaa3f6a6ce955f4168632b967a8
msg323616 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-16 19:38
New changeset cabe916dc694997d4892b58986e73a713d5a2f8d by Christian Heimes (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (#8790)
https://github.com/python/cpython/commit/cabe916dc694997d4892b58986e73a713d5a2f8d
msg323617 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-08-16 19:38
New changeset 2ec530cd5537dfda5ca0af6ac696e45013ed31d2 by Christian Heimes in branch '2.7':
[2.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (GH-8791)
https://github.com/python/cpython/commit/2ec530cd5537dfda5ca0af6ac696e45013ed31d2
msg323678 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-08-17 22:01
Is there more to do on this?  Tests pass on Windows with 1 skip, but I don't know if the failure condition applies here.
test_check_hostname (test.test_ftplib.TestTLS_FTPClass) ... skipped 'FIXME: bpo-32706'
msg391404 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2021-04-19 20:13
I don't think there is anything left to do.
History
Date User Action Args
2021-04-19 20:13:29christian.heimessetstatus: open -> closed
resolution: fixed
messages: + msg391404

stage: patch review -> resolved
2018-08-17 22:01:37terry.reedysetnosy: + terry.reedy
messages: + msg323678
2018-08-16 19:38:46christian.heimessetmessages: + msg323617
2018-08-16 19:38:05christian.heimessetmessages: + msg323616
2018-08-16 19:37:22christian.heimessetmessages: + msg323615
2018-08-16 17:46:49christian.heimessetpull_requests: + pull_request8265
2018-08-16 17:44:25miss-islingtonsetpull_requests: + pull_request8264
2018-08-16 17:44:13miss-islingtonsetpull_requests: + pull_request8263
2018-08-16 17:43:48christian.heimessetmessages: + msg323612
2018-08-16 16:51:26christian.heimessetkeywords: + patch
pull_requests: + pull_request8261
2018-08-16 16:40:02christian.heimessetversions: + Python 2.7, Python 3.6, Python 3.7, Python 3.8
messages: + msg323608

assignee: christian.heimes
components: + SSL
type: behavior
stage: patch review
2018-08-13 09:50:35christian.heimescreate