classification
Title: AttributeError when parsing multipart email with invalid non-decodable Content-Transfer-Encoding
Type: behavior Stage: test needed
Components: email, Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Andrew Donnellan, barry, r.david.murray
Priority: normal Keywords:

Created on 2017-07-03 13:37 by Andrew Donnellan, last changed 2018-12-06 09:36 by serhiy.storchaka.

Files
File name Uploaded Description Edit
testprog.py Andrew Donnellan, 2017-07-03 13:37 Example program that demonstrates the error
Pull Requests
URL Status Linked Edit
PR 2544 open Andrew Donnellan, 2017-07-03 13:50
Messages (2)
msg297584 - (view) Author: Andrew Donnellan (Andrew Donnellan) * Date: 2017-07-03 13:37
Parsing an email containing a multipart Content-Type, along with a Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte will fail.

email.feedparser.FeedParser._parsegen() calls "self._cur.get('content-transfer-encoding', '8bit')" to get the header.

It then tries to check whether the C-T-E is in the allowable set of ('7bit', '8bit', 'binary'), and to do so case-insensitively, it tries to convert the header to lowercase. However, because there's an invalid character in there, it's dealing with a Header object rather than a str. Hence it throws an AttributeError.

Correct behaviour would be to convert the Header to a str, see that it's not valid, and continue on to handle the defect as usual.

Thanks to Daniel Axtens for finding this bug as he was running the AFL fuzzer on the email parsing code in Patchwork (https://github.com/getpatchwork/patchwork).

Pull request incoming.
msg297771 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-05 17:17
There's a deeper problem here involving how Header is used in compat32 that I've been aware of for a while but haven't had time to try to think through a fix for (there may not be one, given the history of the compat32 code).  In the meantime, the proposed fix is reasonable.  (It isn't needed for the new policies, but it doesn't hurt.)
History
Date User Action Args
2018-12-06 09:36:19serhiy.storchakasetstage: test needed
versions: + Python 3.8, - Python 3.5
2017-07-05 17:17:49r.david.murraysetversions: - Python 3.3, Python 3.4
2017-07-05 17:17:32r.david.murraysetmessages: + msg297771
2017-07-03 13:50:12Andrew Donnellansetpull_requests: + pull_request2613
2017-07-03 13:37:03Andrew Donnellancreate