This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Exception parsing invalid email address headers starting or ending with dot
Type: behavior Stage: patch review
Components: email Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Steven Hilton, barry, cnicodeme, iritkatriel, kfogel, ncoghlan, r.david.murray, timb07
Priority: normal Keywords: patch

Created on 2017-07-22 04:50 by timb07, last changed 2022-04-11 14:58 by admin.

Pull Requests
URL Status Linked Edit
PR 2811 open timb07, 2017-07-22 05:02
PR 15600 open tsufeki, 2019-08-29 18:40
PR 18687 closed Steven Hilton, 2020-03-03 09:09
Messages (4)
msg298836 - (view) Author: Tim Bell (timb07) * Date: 2017-07-22 04:50
Email addresses with a display name starting with a dot ("."), or ending with a dot without whitespace before the angle bracket trigger exceptions when accessing the header, after creating the message object with the "default" policy.

For example:

>>> import email
>>> from email.policy import default
>>> email.message_from_bytes(b'To: . Doe <jxd@example.com>')['to']
'. Doe <jxd@example.com>'
>>> email.message_from_bytes(b'To: . Doe <jxd@example.com>', policy=default)['to']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bhat/git/cpython/Lib/email/message.py", line 391, in __getitem__
    return self.get(name)
  File "/Users/bhat/git/cpython/Lib/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/Users/bhat/git/cpython/Lib/email/policy.py", line 162, in header_fetch_parse
    return self.header_factory(name, value)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 586, in __call__
    return self[name](name, value)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 344, in parse
    for mb in addr.all_mailboxes]))
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 344, in <listcomp>
    for mb in addr.all_mailboxes]))
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 834, in display_name
    return self[0].display_name
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 768, in display_name
    return self[0].display_name
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 931, in display_name
    if res[0][0].token_type == 'cfws':
AttributeError: 'str' object has no attribute 'token_type'
>>>
>>> email.message_from_bytes(b'To: John X.<jxd@example.com>')['to']
'John X.<jxd@example.com>'
>>> email.message_from_bytes(b'To: John X.<jxd@example.com>', policy=default)['to']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bhat/git/cpython/Lib/email/message.py", line 391, in __getitem__
    return self.get(name)
  File "/Users/bhat/git/cpython/Lib/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/Users/bhat/git/cpython/Lib/email/policy.py", line 162, in header_fetch_parse
    return self.header_factory(name, value)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 586, in __call__
    return self[name](name, value)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 344, in parse
    for mb in addr.all_mailboxes]))
  File "/Users/bhat/git/cpython/Lib/email/headerregistry.py", line 344, in <listcomp>
    for mb in addr.all_mailboxes]))
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 834, in display_name
    return self[0].display_name
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 768, in display_name
    return self[0].display_name
  File "/Users/bhat/git/cpython/Lib/email/_header_value_parser.py", line 936, in display_name
    if res[-1][-1].token_type == 'cfws':
AttributeError: 'str' object has no attribute 'token_type'
msg357638 - (view) Author: Cyril Nicodème (cnicodeme) Date: 2019-11-29 11:04
Hi!

I confirm this problem too, also with the SMTPUTF8 policy.

I was able to reproduce this error on my end (Python v3.7.5).

Note that when calling `message_from_bytes` without policy, there is no errors.
msg383564 - (view) Author: Karl Fogel (kfogel) Date: 2020-12-22 01:39
I can also confirm this bug, with Python 3.9.1 on Debian GNU/Linux ('testing' distro up-to-date as of 2020-12-21).

1) Create a parser `p` with `p = email.parser.HeaderParser(policy=email.policy.default)`.

2) Parse a single problematic (as described below) message: `msg_headers = p.parsestr(msg_str)`

3) Try to get the To field: `msg_headers.get_all('to', [ ])` and get an exception raised: `AttributeError: 'str' object has no attribute 'token_type'` (with the same stack trace as OP shows).

Here is a minimal problematic message you can reproduce this with (i.e., just make `msg_str` have this string value):

```
From nobody Mon Dec 21 12:00:00  2020
From: sender@example.com
To: . <jrandom@example.com>
Date: Mon, 21 Dec 2020 12:00:00 -0000
Message-ID: <87ab5rvds7.fsf@example.com>
Subject: This is the Subject header.

Here is the body of the message.
```

Note that *any* number of dots for the recipient's name would also result in an error.  The above example uses just ".", but it could be "..", "...", ".................", etc.
msg407789 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-12-06 11:29
Reproduced on 3.11.
History
Date User Action Args
2022-04-11 14:58:49adminsetgithub: 75171
2021-12-06 11:29:24iritkatrielsetnosy: + iritkatriel

messages: + msg407789
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 3.5, Python 3.6, Python 3.7
2020-12-22 01:39:34kfogelsetnosy: + kfogel
messages: + msg383564
2020-03-03 09:09:55Steven Hiltonsetnosy: + Steven Hilton
pull_requests: + pull_request18121
2019-11-29 11:04:39cnicodemesetnosy: + cnicodeme
messages: + msg357638
2019-08-29 18:40:06tsufekisetkeywords: + patch
stage: patch review
pull_requests: + pull_request15276
2017-08-08 01:34:39ncoghlansetnosy: + ncoghlan
2017-07-22 05:02:10timb07setpull_requests: + pull_request2862
2017-07-22 04:50:02timb07create