classification
Title: AttributeError when parsing multipart email with invalid non-decodable Content-Transfer-Encoding
Type: behavior Stage: patch review
Components: email, Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Andrew Donnellan, Jeffrey.Kintscher, barry, miss-islington, r.david.murray
Priority: normal Keywords: patch

Created on 2017-07-03 13:37 by Andrew Donnellan, last changed 2019-06-05 10:23 by miss-islington.

Files
File name Uploaded Description Edit
testprog.py Andrew Donnellan, 2017-07-03 13:37 Example program that demonstrates the error
Pull Requests
URL Status Linked Edit
PR 2544 closed Andrew Donnellan, 2017-07-03 13:50
PR 13598 merged maxking, 2019-05-27 22:34
PR 13820 merged miss-islington, 2019-06-04 18:01
PR 13821 closed miss-islington, 2019-06-04 18:01
Messages (4)
msg297584 - (view) Author: Andrew Donnellan (Andrew Donnellan) * Date: 2017-07-03 13:37
Parsing an email containing a multipart Content-Type, along with a Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte will fail.

email.feedparser.FeedParser._parsegen() calls "self._cur.get('content-transfer-encoding', '8bit')" to get the header.

It then tries to check whether the C-T-E is in the allowable set of ('7bit', '8bit', 'binary'), and to do so case-insensitively, it tries to convert the header to lowercase. However, because there's an invalid character in there, it's dealing with a Header object rather than a str. Hence it throws an AttributeError.

Correct behaviour would be to convert the Header to a str, see that it's not valid, and continue on to handle the defect as usual.

Thanks to Daniel Axtens for finding this bug as he was running the AFL fuzzer on the email parsing code in Patchwork (https://github.com/getpatchwork/patchwork).

Pull request incoming.
msg297771 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-05 17:17
There's a deeper problem here involving how Header is used in compat32 that I've been aware of for a while but haven't had time to try to think through a fix for (there may not be one, given the history of the compat32 code).  In the meantime, the proposed fix is reasonable.  (It isn't needed for the new policies, but it doesn't hurt.)
msg344620 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2019-06-04 18:01
New changeset aa79707262f893428665ef45b5e879129abca4aa by Barry Warsaw (Abhilash Raj) in branch 'master':
bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
https://github.com/python/cpython/commit/aa79707262f893428665ef45b5e879129abca4aa
msg344698 - (view) Author: miss-islington (miss-islington) Date: 2019-06-05 10:23
New changeset f62a372928fbf6a2ba722f12f069b75ca6ad16fb by Miss Islington (bot) in branch '3.7':
bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
https://github.com/python/cpython/commit/f62a372928fbf6a2ba722f12f069b75ca6ad16fb
History
Date User Action Args
2019-06-05 10:23:41miss-islingtonsetnosy: + miss-islington
messages: + msg344698
2019-06-04 18:01:10miss-islingtonsetpull_requests: + pull_request13707
2019-06-04 18:01:04barrysetmessages: + msg344620
2019-06-04 18:01:00miss-islingtonsetpull_requests: + pull_request13706
2019-05-31 05:34:07Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher
2019-05-27 22:34:44maxkingsetkeywords: + patch
stage: test needed -> patch review
pull_requests: + pull_request13505
2018-12-06 09:36:19serhiy.storchakasetstage: test needed
versions: + Python 3.8, - Python 3.5
2017-07-05 17:17:49r.david.murraysetversions: - Python 3.3, Python 3.4
2017-07-05 17:17:32r.david.murraysetmessages: + msg297771
2017-07-03 13:50:12Andrew Donnellansetpull_requests: + pull_request2613
2017-07-03 13:37:03Andrew Donnellancreate