classification
Title: imaplib: unlimited readline() from connection
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Arfrever, Emil.Lind, akuchling, barry, benjamin.peterson, christian.heimes, georg.brandl, giampaolo.rodola, haypo, larry, python-dev, r.david.murray
Priority: release blocker Keywords: patch

Created on 2012-09-25 10:36 by christian.heimes, last changed 2014-09-30 14:28 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
imaplib.issue16039.patch Emil.Lind, 2013-02-26 22:10 review
imaplib.txt akuchling, 2013-09-15 19:53
Messages (27)
msg171242 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-09-25 10:36
This bug is similar to #16037 and a modified copy of #16038.

The imaplib module doesn't limit the amount of read data in its call to readline(). An erroneous or malicious IMAP server can trick the imaplib module to consume large amounts of memory.

Suggestion:
The imaplib module should be modified to use limited readline() with _MAXLINE like the httplib module.
msg182193 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-15 23:53
RFC 3501 and 2060 (IMAP 4rev1) don't specify a line length


RFC 2683 says:

  A client should limit the length of the command lines it
  generates to approximately 1000 octets.

  For its part, a server should allow for a command line of at
  least 8000 octets.

Some config files and code have values between 2k and 64k, usually around 8k to 10k, e.g.

 UW and Panda IMAP have a limit of 10,000 octets which is far
 more than what anything is ever likely to use.
msg182196 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-15 23:59
CVE-2013-1752  Unbound readline() DoS vulnerabilities in Python stdlib
msg183096 - (view) Author: Emil Lind (Emil.Lind) * Date: 2013-02-26 22:10
I'm uploading my first patch. 
Heavily based on the related issues for ftplib and poplib.
Need help with review and a few questions...

Q1: Is the error Exception the right way to handle the "breach" (disconnects client?) or is there a better way? Like a 'BAD' response...

Q2: I'm not sure how to best modify the test_imaplib for this patch. I'm guessing a make_server where the client gets MAXLINE+1 bytes of data and validates exception. But it's above my abilities right now...

I welcome any input, thanks. 

note: patch seems to apply to 2.7, 3.2, 3.3, 3.4
msg185054 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-03-23 14:45
Not blocking 2.7.4 as discussed on mailing list.
msg196860 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-09-03 18:34
blocker for 2.6.9
msg197823 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2013-09-15 19:53
Updated version of the patch against 2.6 that adds a test.  Thanks for the fix, Emil!
msg198299 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-09-22 20:03
Looks good for 2.6.  The NEWS file hunk doesn't apply, but I'll fix that when I commit this to 2.6.
msg198300 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-22 21:15
New changeset 4190568ceda0 by Barry Warsaw in branch '2.6':
- Issue #16039: CVE-2013-1752: Change use of readline in imaplib module to
http://hg.python.org/cpython/rev/4190568ceda0
msg198301 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-09-22 21:17
Since the merge 2.6 -> 2.7 did not apply cleanly, and had other problems. I null merged the 2.6 changes.  I'll leave it to Benjamin to work out whatever patches 2.7 needs.
msg200346 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-10-19 01:20
Ping.  Please fix before "beta 1".
msg201426 - (view) Author: Roundup Robot (python-dev) Date: 2013-10-27 06:39
New changeset 4b0364fc5711 by Georg Brandl in branch '3.3':
Issue #16039: CVE-2013-1752: Change use of readline in imaplib module to limit
http://hg.python.org/cpython/rev/4b0364fc5711
msg201427 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-27 06:45
Also merged to default.
msg207218 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-01-03 10:47
Why is this issue still open? The issue was fixed in Python 2.6.9. Why is the issue a release blocker? The issue was also fixed in the future Python 3.4 (in default).
msg207224 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-03 12:24
Presumably because it has not been fixed in 2.7.
msg207227 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-01-03 13:13
"Since the merge 2.6 -> 2.7 did not apply cleanly, and had other problems. I null merged the 2.6 changes.  I'll leave it to Benjamin to work out whatever patches 2.7 needs."

So Benjamin, is there a reason to not fix this security vulnerability in Python 2.7?
msg207231 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-01-03 16:57
There's no reason not to fix it assuming the patch is good...
msg207233 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-03 19:02
Applied to 2.7 in dd906f4ab923.
msg207239 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-03 19:42
And we're getting test failures in the SSL version of the test.  No similar failure reports in the tracker, and the same test has been running on the Python3 branch for a while now.
msg207254 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-03 22:27
New changeset d7ae948d9eee by R David Murray in branch '2.7':
#16039/#20118: temporarily skip failing imaplib SSL test.
http://hg.python.org/cpython/rev/d7ae948d9eee
msg207255 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-01-03 22:34
Reopen, a test is failing.
msg207256 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-01-03 23:00
I opened a new issue for the failing test: issue 20118, so I don't see a reason to keep this open.
msg207257 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-01-03 23:21
"I opened a new issue for the failing test: issue 20118, so I don't see a reason to keep this open."

Ok, I wasn't aware of this issue.
msg227925 - (view) Author: Roundup Robot (python-dev) Date: 2014-09-30 14:02
New changeset 5d1c03316af7 by Georg Brandl in branch '3.2':
Issue #16039: CVE-2013-1752: Change use of readline in imaplib module to limit
https://hg.python.org/cpython/rev/5d1c03316af7
msg227927 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-09-30 14:11
> New changeset 5d1c03316af7 by Georg Brandl in branch '3.2':
> Issue #16039: CVE-2013-1752: Change use of readline in imaplib module to limit
> https://hg.python.org/cpython/rev/5d1c03316af7

I'm not sure that this change is correct, the test failed on Windows. Or maybe, it's just an issue with test test?

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/5168/steps/test/logs/stdio

======================================================================
ERROR: test_connect (test.test_smtpnet.SmtpSSLTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_smtpnet.py", line 16, in test_connect
    server = smtplib.SMTP_SSL(self.testServer, self.remotePort)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\smtplib.py", line 862, in __init__
    SMTP.__init__(self, host, port, local_hostname, timeout)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\smtplib.py", line 260, in __init__
    (code, msg) = self.connect(host, port)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\smtplib.py", line 321, in connect
    (code, msg) = self.getreply()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\smtplib.py", line 367, in getreply
    line = self.file.readline(_MAXLINE + 1)
TypeError: readline() takes exactly 1 positional argument (2 given)
msg227928 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-09-30 14:14
Let me check that.
msg227931 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-09-30 14:24
This error is rather related to issue #16042, not issue #16039.
History
Date User Action Args
2014-09-30 14:28:15georg.brandlsetstatus: open -> closed
resolution: fixed
2014-09-30 14:24:12Arfreversetmessages: + msg227931
2014-09-30 14:14:12georg.brandlsetmessages: + msg227928
2014-09-30 14:11:08hayposetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg227927
2014-09-30 14:04:41georg.brandlsetversions: - Python 3.1, Python 3.2
2014-09-30 14:02:26python-devsetmessages: + msg227925
2014-01-03 23:21:33hayposetstatus: open -> closed
resolution: fixed
messages: + msg207257
2014-01-03 23:00:36r.david.murraysetmessages: + msg207256
2014-01-03 22:34:44hayposetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg207255
2014-01-03 22:27:32python-devsetmessages: + msg207254
2014-01-03 19:42:00r.david.murraysetmessages: + msg207239
2014-01-03 19:02:15r.david.murraysetstatus: open -> closed
resolution: fixed
messages: + msg207233

stage: needs patch -> resolved
2014-01-03 16:57:40benjamin.petersonsetmessages: + msg207231
2014-01-03 13:13:47hayposetmessages: + msg207227
2014-01-03 12:24:41r.david.murraysetnosy: + r.david.murray
messages: + msg207224
2014-01-03 10:47:59hayposetnosy: + haypo
messages: + msg207218
2013-10-27 06:45:40georg.brandlsetmessages: + msg201427
versions: - Python 3.3, Python 3.4
2013-10-27 06:39:05python-devsetmessages: + msg201426
2013-10-19 01:20:44larrysetmessages: + msg200346
2013-09-22 21:17:04barrysetmessages: + msg198301
versions: - Python 2.6
2013-09-22 21:15:31python-devsetnosy: + python-dev
messages: + msg198300
2013-09-22 20:03:33barrysetmessages: + msg198299
2013-09-15 19:53:52akuchlingsetfiles: + imaplib.txt
nosy: + akuchling
messages: + msg197823

2013-09-15 19:43:31Arfreversetversions: + Python 2.6, Python 3.1
2013-09-03 18:34:33barrysetpriority: critical -> release blocker
nosy: + barry
messages: + msg196860

2013-03-23 14:45:17benjamin.petersonsetpriority: release blocker -> critical

messages: + msg185054
2013-02-26 22:10:52Emil.Lindsetfiles: + imaplib.issue16039.patch

nosy: + Emil.Lind
messages: + msg183096

keywords: + patch
2013-02-22 23:47:25Arfreversetnosy: + Arfrever
2013-02-15 23:59:06christian.heimessetmessages: + msg182196
2013-02-15 23:53:50christian.heimessetmessages: + msg182193
2013-02-04 17:12:34christian.heimessetpriority: critical -> release blocker
nosy: + georg.brandl, benjamin.peterson, larry
2013-01-21 11:37:11giampaolo.rodolasetnosy: + giampaolo.rodola
2013-01-20 14:38:25christian.heimessetpriority: normal -> critical
assignee: christian.heimes
stage: needs patch
versions: + Python 2.7, Python 3.2, Python 3.3, Python 3.4
2012-09-25 10:36:10christian.heimescreate