classification
Title: test_imaplib test_linetoolong fails on 2.7 in SSL test on some buildbots
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, pablogsal, python-dev, r.david.murray, vstinner
Priority: normal Keywords: patch

Created on 2014-01-03 22:22 by r.david.murray, last changed 2018-12-12 15:06 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 11120 merged vstinner, 2018-12-11 13:53
Messages (7)
msg207252 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-03 22:22
test.test_imaplib.ThreadedNetworkedTestsSSL.test_linetoolong, added as part of the fix for security (DOS attack) issue 16039, is failing on 2.7 on some buildbots (FreeBSD, OpenIndia, Windows).  On some platforms (FreeBSD at least) the failure seems to be intermittent.  Running the tests with the same randseed on linux does not reproduce the problem.  Running the test under -F on linux does not produce any failures.

Since this is a DOS security issue, and since the failure is that the exception comes from the ssl module instead of being an imaplib error but it nevertheless blocks the attack, I've left the patch in.  But the reason for the failure should be determined and either the test or the code fixed.  (In the meantime I'll mark the test to be skipped).
msg207253 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-01-03 22:27
New changeset d7ae948d9eee by R David Murray in branch '2.7':
#16039/#20118: temporarily skip failing imaplib SSL test.
http://hg.python.org/cpython/rev/d7ae948d9eee
msg331624 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-11 13:49
4 years later the test is still skipped. That's a good example of "temporary fix" :-)

David: Should we just close the issue, or is there anyone around interested to work on a fix? Sadly, the issue doesn't provide the error message and I'm not sure how to reproduce the bug.

Maybe the bug has been fixed in the meanwhile? Example of SocketServer fix:

commit ba8474b77dd86d8dde40eaa7a4a6715a476d6242
Author: Martin Panter <vadmium+py@gmail.com>
Date:   Thu Feb 18 10:43:55 2016 +0000

    Issue #26309: Shut down SocketServer request if verify_request() is false
    
    Based on patch by Aviv Palivoda.

--

On Fedora 29, I tried to reproduce the bug by running many tests in parallel:

(*) Run test_linetoolong in a loop, 4 processes:

./python -m test -u all -F -m test.test_imaplib.ThreadedNetworkedTests.test_linetoolong test_imaplib 

(*) Run test_imaplib in a loop, 4 processes:

./python -m test -u all -v -F test_imaplib

(*) Stress the system (run the Python test suite in a loop), 2 processes:

./python -m test -r -u all,-gui -j0 -F

(*) Stress the CPU using my script (to ensure that the system load is at least 10)

system_load.py 10

7 minutes later, test_linetoolong has been run 11629 times x 4 processes and it's still pass.
msg331625 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-11 13:54
I wrote PR 11120 to reenable the skipped test. If the test will start again, I will try to fix it. If I cannot fix the test, I will skip again the test but close this issue.
msg331630 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-11 14:30
Oh, AppVeyor failed on my PR:

test_linetoolong (test.test_imaplib.ThreadedNetworkedTestsSSL) ...
creating server
server created
ADDR = ('127.0.0.1', 0)
CLASS = test.test_imaplib.SecureTCPServer
HDLR = test.test_imaplib.TooLongHandler
server running
waiting for server
done
ERROR

(...)

ERROR: test_linetoolong (test.test_imaplib.ThreadedNetworkedTestsSSL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_imaplib.py", line 176, in test_linetoolong
    self.imap_class, *server.server_address)
  File "C:\projects\cpython\lib\unittest\case.py", line 473, in assertRaises
    callableObj(*args, **kwargs)
  File "C:\projects\cpython\lib\imaplib.py", line 1169, in __init__
    IMAP4.__init__(self, host, port)
  File "C:\projects\cpython\lib\imaplib.py", line 203, in __init__
    typ, dat = self.capability()
  File "C:\projects\cpython\lib\imaplib.py", line 377, in capability
    typ, dat = self._simple_command(name)
  File "C:\projects\cpython\lib\imaplib.py", line 1091, in _simple_command
    return self._command_complete(name, self._command(name, *args))
  File "C:\projects\cpython\lib\imaplib.py", line 913, in _command_complete
    typ, data = self._get_tagged_response(tag)
  File "C:\projects\cpython\lib\imaplib.py", line 1020, in _get_tagged_response
    self._get_response()
  File "C:\projects\cpython\lib\imaplib.py", line 932, in _get_response
    resp = self._get_line()
  File "C:\projects\cpython\lib\imaplib.py", line 1030, in _get_line
    line = self.readline()
  File "C:\projects\cpython\lib\imaplib.py", line 1192, in readline
    return self.file.readline()
  File "C:\projects\cpython\lib\socket.py", line 451, in readline
    data = self._sock.recv(self._rbufsize)
  File "C:\projects\cpython\lib\ssl.py", line 772, in recv
    return self.read(buflen)
  File "C:\projects\cpython\lib\ssl.py", line 659, in read
    v = self._sslobj.read(len)
error: [Errno 10053] An established connection was aborted by the software in your host machine
msg331691 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-12 11:06
New changeset 16d63202af35dadd652a5e3eae687ea709e95b11 by Victor Stinner in branch '2.7':
bpo-16039: CVE-2013-1752: Limit imaplib.IMAP4_SSL.readline() (GH-11120)
https://github.com/python/cpython/commit/16d63202af35dadd652a5e3eae687ea709e95b11
msg331697 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-12-12 15:06
The issue should now be fixed. I will reopen it if the test starts failing on a CI.
History
Date User Action Args
2018-12-12 15:06:55vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg331697

stage: patch review -> resolved
2018-12-12 11:06:12vstinnersetmessages: + msg331691
2018-12-11 14:30:29vstinnersetmessages: + msg331630
2018-12-11 13:54:18vstinnersetmessages: + msg331625
2018-12-11 13:53:25vstinnersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request10350
2018-12-11 13:49:47vstinnersetnosy: + pablogsal, vstinner
messages: + msg331624
2014-01-04 06:57:30Arfreversetnosy: + Arfrever
2014-01-03 22:27:31python-devsetnosy: + python-dev
messages: + msg207253
2014-01-03 22:22:59r.david.murraycreate