This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: poplib: unlimited readline() from connection
Type: resource usage Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: christian.heimes Nosy List: Arfrever, Chris Smowton, Ingo Ruhnke, barry, benjamin.peterson, berker.peksag, christian.heimes, doko, georg.brandl, giampaolo.rodola, larry, nailor, python-dev, r.david.murray, scoulson
Priority: release blocker Keywords: patch

Created on 2012-09-25 10:39 by christian.heimes, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue16041.patch nailor, 2013-02-23 19:19
issue16041_py26.patch nailor, 2013-09-29 19:57 review
issue16041_py26_with_ssl.patch nailor, 2013-09-30 17:33 review
issue16041_py32.patch nailor, 2013-10-24 19:07 review
issue16041_27.diff berker.peksag, 2014-10-19 14:36
Messages (23)
msg171244 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-09-25 10:39
This bug is similar to #16037 and a modified copy of #16038.

The poplib module doesn't limit the amount of read data in its call to readline(). An erroneous or malicious POP3 server can trick the poplib module to consume large amounts of memory.

Suggestion:
The poplib module should be modified to use limited readline() with _MAXLINE like the httplib module.
msg182189 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-15 23:24
RFC 1939 says:

   Responses in the POP3 consist of a status indicator and a keyword
   possibly followed by additional information.  All responses are
   terminated by a CRLF pair.  Responses may be up to 512 characters
   long, including the terminating CRLF.

It doesn't say anything about the length of a line in a multi-line response. It's reasonable to belief that 512 octets are valid, too. We could quadruple the limit to 2048 in order to be safe.
msg182198 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2013-02-15 23:59
CVE-2013-1752  Unbound readline() DoS vulnerabilities in Python stdlib
msg182797 - (view) Author: Jyrki Pulliainen (nailor) * Date: 2013-02-23 19:19
Added a functionality that raises error_proto('line too long') if we read over _MAXLINE characters. Defaults _MAXLINE to 2048. The patch is written on top of 2.7
msg185056 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-03-23 14:45
Not blocking 2.7.4 as discussed on mailing list.
msg198639 - (view) Author: Jyrki Pulliainen (nailor) * Date: 2013-09-29 19:57
Attached a patch for 2.6. 

Due to how tests are in 2.6, this adds one more test case with evil server, which basically just returns too long lines.
msg198651 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-09-29 21:05
Looks good, although only the POP3 exception is actually tested.  The POP3_SSL exception isn't tested.  Any chance you could add a test for that (obviously, only if `import ssl` succeeds)?
msg198710 - (view) Author: Jyrki Pulliainen (nailor) * Date: 2013-09-30 17:33
Added a test for SSL, if SSL is available
msg198727 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-09-30 19:58
New changeset 7214e3324a45 by Barry Warsaw in branch '2.6':
- Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
http://hg.python.org/cpython/rev/7214e3324a45
msg198737 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2013-09-30 20:41
> New changeset 7214e3324a45 by Barry Warsaw in branch '2.6':
> - Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
> http://hg.python.org/cpython/rev/7214e3324a45
> ...
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> ...
> +- Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
> +  prevent readline() calls from consuming too much member.

Maybe s/member/memory/ ?
msg198738 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-09-30 20:47
On Sep 30, 2013, at 08:41 PM, Arfrever Frehtes Taifersar Arahesis wrote:

>
>Arfrever Frehtes Taifersar Arahesis added the comment:
>
>> New changeset 7214e3324a45 by Barry Warsaw in branch '2.6':
>> - Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
>> http://hg.python.org/cpython/rev/7214e3324a45
>> ...
>> --- a/Misc/NEWS
>> +++ b/Misc/NEWS
>> ...
>> +- Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
>> +  prevent readline() calls from consuming too much member.
>
>Maybe s/member/memory/ ?

Good catch, thanks.
msg200350 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-10-19 01:23
Ping.  Please fix before "beta 1".
msg201164 - (view) Author: Jyrki Pulliainen (nailor) * Date: 2013-10-24 19:07
Here's a max line lenght fix for 3.2 (applies on 3.4 too).

I wonder if _getlongresp should have some max length detection too for max length of a multiline response
msg201423 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-27 06:38
New changeset 68029048c9c6 by Georg Brandl in branch '3.3':
Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
http://hg.python.org/cpython/rev/68029048c9c6
msg201430 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-27 06:46
Also merged to default.
msg227891 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-09-30 12:47
New changeset 76be07730f8d by Georg Brandl in branch '3.2':
Issue #16041: CVE-2013-1752: poplib: Limit maximum line lengths to 2048 to
https://hg.python.org/cpython/rev/76be07730f8d
msg229685 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-10-19 14:07
Here is a patch for 2.7.
msg231988 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-12-02 11:54
this looks ok to me, can we apply this for 2.7.9?
msg232230 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-12-06 01:17
New changeset 339f877cca11 by Benjamin Peterson in branch '2.7':
in poplib, limit maximum line length that we read from the network (closes #16041)
https://hg.python.org/cpython/rev/339f877cca11
msg245901 - (view) Author: Ingo Ruhnke (Ingo Ruhnke) Date: 2015-06-28 06:48
This "fix" has broken mail retrieval from both gmx.de, gmail.com and plenty of other provider.

It manifests in getmail as:

Retrieval error: server for BrokenUIDLPOP3SSLRetriever:1860228@pop.gmx.net:995 is broken; offered message 239 but failed to provide it.  Please notify the administrator of the server.  Skipping message...

After setting the _MAXLINE in /usr/lib/python2.7/poplib.py to something higher everything was working again.

See issue #23906
msg246727 - (view) Author: Chris Smowton (Chris Smowton) Date: 2015-07-14 11:03
+1 to the above; suggest this should be rolled back and replaced with a total message size limit.
msg248438 - (view) Author: Stephen Coulson (scoulson) Date: 2015-08-12 01:22
Broke for me today. Hacked the _MAXLINE to get around it.
I don't see any size limit on multi-line in rfc. Only requirement is dot-stuffing. I think this fix might need a rethink.
msg248454 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-08-12 11:44
It has been, see the referenced issue.  Now we just need someone to write a patch.
History
Date User Action Args
2022-04-11 14:57:36adminsetgithub: 60245
2015-08-12 11:44:04r.david.murraysetnosy: + r.david.murray
messages: + msg248454
2015-08-12 01:22:05scoulsonsetnosy: + scoulson
messages: + msg248438
2015-07-14 11:03:13Chris Smowtonsetnosy: + Chris Smowton
messages: + msg246727
2015-06-28 06:48:32Ingo Ruhnkesetnosy: + Ingo Ruhnke
messages: + msg245901
2014-12-06 01:17:14python-devsetstatus: open -> closed
resolution: fixed
messages: + msg232230

stage: patch review -> resolved
2014-12-02 11:54:50dokosetnosy: + doko
messages: + msg231988
2014-10-19 14:37:09berker.peksagsetfiles: - issue16041_27.diff
2014-10-19 14:36:45berker.peksagsetfiles: + issue16041_27.diff
2014-10-19 14:07:47berker.peksagsetfiles: + issue16041_27.diff

nosy: + berker.peksag
messages: + msg229685

stage: needs patch -> patch review
2014-09-30 12:49:47georg.brandlsetversions: - Python 3.2
2014-09-30 12:47:29python-devsetmessages: + msg227891
2014-09-30 12:37:24georg.brandlsetversions: - Python 3.1
2013-10-27 06:46:08georg.brandlsetmessages: + msg201430
versions: - Python 3.3, Python 3.4
2013-10-27 06:38:45python-devsetmessages: + msg201423
2013-10-24 19:07:27nailorsetfiles: + issue16041_py32.patch

messages: + msg201164
2013-10-19 01:23:04larrysetmessages: + msg200350
2013-09-30 20:47:27barrysetmessages: + msg198738
2013-09-30 20:41:49Arfreversetmessages: + msg198737
2013-09-30 20:00:03barrysetversions: - Python 2.6
2013-09-30 19:58:47python-devsetnosy: + python-dev
messages: + msg198727
2013-09-30 17:33:11nailorsetfiles: + issue16041_py26_with_ssl.patch

messages: + msg198710
2013-09-29 21:05:20barrysetmessages: + msg198651
2013-09-29 19:57:09nailorsetfiles: + issue16041_py26.patch

messages: + msg198639
2013-09-15 19:45:28Arfreversetpriority: critical -> release blocker
nosy: + barry

versions: + Python 2.6, Python 3.1
2013-03-23 14:45:29benjamin.petersonsetpriority: release blocker -> critical

messages: + msg185056
2013-02-23 19:19:38nailorsetfiles: + issue16041.patch

nosy: + nailor
messages: + msg182797

keywords: + patch
2013-02-22 23:48:13Arfreversetnosy: + Arfrever
2013-02-15 23:59:23christian.heimessetmessages: + msg182198
2013-02-15 23:24:06christian.heimessetmessages: + msg182189
2013-02-04 17:12:41christian.heimessetpriority: critical -> release blocker
nosy: + georg.brandl, benjamin.peterson, larry
2013-01-21 11:37:27giampaolo.rodolasetnosy: + giampaolo.rodola
2013-01-20 14:37:35christian.heimessetpriority: normal -> critical
assignee: christian.heimes
stage: needs patch
versions: + Python 3.4
2012-09-25 10:39:22christian.heimescreate