This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Certain Malformed email causes email.parser to throw AttributeError
Type: behavior Stage: needs patch
Components: email Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeffrey.Kintscher, barry, matrixise, maxking, msapiro, r.david.murray
Priority: normal Keywords:

Created on 2019-05-14 01:48 by msapiro, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
parse_bug.py msapiro, 2019-05-14 01:48 Python code that triggers bug
Messages (8)
msg342415 - (view) Author: Mark Sapiro (msapiro) * (Python triager) Date: 2019-05-14 01:48
The code in the attached parse_bug.py file when run with Python 3.5, 3.6 or 3.7 throws AttributeError with this traceback:

```
Traceback (most recent call last):
  File "parse_bug.py", line 9, in <module>
    """)
  File "/usr/local/lib/python3.7/email/parser.py", line 124, in parsebytes
    return self.parser.parsestr(text, headersonly)
  File "/usr/local/lib/python3.7/email/parser.py", line 68, in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
  File "/usr/local/lib/python3.7/email/parser.py", line 58, in parse
    return feedparser.close()
  File "/usr/local/lib/python3.7/email/feedparser.py", line 187, in close
    self._call_parse()
  File "/usr/local/lib/python3.7/email/feedparser.py", line 180, in _call_parse
    self._parse()
  File "/usr/local/lib/python3.7/email/feedparser.py", line 323, in _parsegen
    if (self._cur.get('content-transfer-encoding', '8bit').lower()
AttributeError: 'Header' object has no attribute 'lower'
```

The triggering condition appears to be the Content-Transfer-Encoding: header with a non-ascii character in the headers of a multipart part.

The parser should probably throw email.errors.HeaderParseError with a MalformedHeaderDefect in this case rather than AttributeError.

While arguably code should defend against unanticipated exceptions, the fact that such an exception can be thrown while parsing an arbitrary message could be considered a security issue.
msg342446 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2019-05-14 10:07
Hi Mark,

Do you want to submit a PR for this issue?
msg342466 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2019-05-14 14:34
Not a security issue, no.  This isn't C where a stack overflow can give an attacker a vector for injecting arbitrary code.

Per the Parser contract ("raise no exceptions, only register defects"), this should, as you say, register a defect (email.errors.InvalidMultipartContentTransferEncodingDefect) and assume a CTE of 7bit for the rest of the parsing.  The problem here is that the feedparser is running into the "hack" I put in place in python3.2 for dealing with invalid binary data in headers (which is to turn it into a Header with charset unknown-8bit).  That works most of the time, but in cases like this it breaks down :(

Note that the new API (policy=default and friends) handles this without error.
msg342480 - (view) Author: Mark Sapiro (msapiro) * (Python triager) Date: 2019-05-14 15:59
I do intend to submit a PR. I haven't yet worked it out though.
msg344018 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2019-05-31 04:26
This is a dupe of https://bugs.python.org/issue30835, which has an attached PR.

I have verified the provided test case is fixed by the PR for bpo-30835.
msg344021 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-31 04:38
The PR for bpo-30835 changes code in a different source file. I suggest adopting both PRs so that future changes don't accidentally reintroduce either bug.
msg344025 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2019-05-31 05:32
I am not sure which 2nd PR are we talking about here?

The reported exception stems from Lib/email/feedparser.py#L323 and same is fixed in PR (https://github.com/python/cpython/pull/13598) for bpo-30835.
msg344028 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-31 06:07
My mistake for responding to the wrong issue.
History
Date User Action Args
2022-04-11 14:59:15adminsetgithub: 81091
2019-06-01 16:33:24berker.peksaglinkissue36976 superseder
2019-05-31 06:07:00Jeffrey.Kintschersetmessages: + msg344028
2019-05-31 05:32:37maxkingsetmessages: + msg344025
2019-05-31 04:38:25Jeffrey.Kintschersetmessages: + msg344021
2019-05-31 04:26:35maxkingsetnosy: + maxking
messages: + msg344018
2019-05-28 08:35:48Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher
2019-05-14 15:59:28msapirosetmessages: + msg342480
2019-05-14 14:34:28r.david.murraysetmessages: + msg342466
2019-05-14 10:08:16SilentGhostsetstage: needs patch
versions: + Python 3.8, - Python 3.5, Python 3.6
2019-05-14 10:07:05matrixisesetnosy: + matrixise
messages: + msg342446
2019-05-14 01:48:39msapirocreate