classification
Title: generated email message exceeds RFC-mandated limit of 998 characters
Type: behavior Stage: resolved
Components: email Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: barry, ivyl, maxking, miss-islington, msapiro, r.david.murray
Priority: normal Keywords: patch

Created on 2020-05-11 19:00 by ivyl, last changed 2020-07-09 00:19 by maxking. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 20038 merged ivyl, 2020-05-11 19:10
PR 20084 merged miss-islington, 2020-05-14 00:53
PR 20542 merged msapiro, 2020-05-30 19:55
PR 21404 merged miss-islington, 2020-07-08 21:00
PR 21405 merged miss-islington, 2020-07-08 21:01
Messages (11)
msg368641 - (view) Author: Arkadiusz Hiler (ivyl) * Date: 2020-05-11 19:00
RFC5322[0] in section 2.1.1 mandates that the line cannot be longer than 998 characters and should not be longer than 78 characters (excluding CRLF). When we use raw_data_manager (default for EmailPolicy, EmailMessage) it does the correct thing as long as the message contains characters outside of 7bit US-ASCII set - base64 or qp Content-Transfer-Encoding which respects the line wrapping at 78 characters.

However if our message is limited the characters from the 7bit US-ASCII set no transfer encoding is applied, and such messages can easily go beyond 78 or even 998 characters.


[0]: https://tools.ietf.org/html/rfc5322.html#section-2.1.1
msg368646 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2020-05-11 20:43
The PR looks good to me, but I describe the change differently.  I'm not sure how I missed this in the original implementation, since I obviously checked it for the 8bit case.  Too long ago to remember :)
msg368810 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2020-05-14 00:53
New changeset 6f2f475d5a2cd7675dce844f3af436ba919ef92b by Arkadiusz Hiler in branch 'master':
bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038)
https://github.com/python/cpython/commit/6f2f475d5a2cd7675dce844f3af436ba919ef92b
msg368811 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2020-05-14 00:57
Thanks, Arkadiusz.
msg369157 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2020-05-18 00:57
New changeset c1f1ddf30a595c2bfa3c06e54fb03fa212cd28b5 by Miss Islington (bot) in branch '3.8':
bpo-40597: email: Use CTE if lines are longer than max_line_length consistently (gh-20038) (gh-20084)
https://github.com/python/cpython/commit/c1f1ddf30a595c2bfa3c06e54fb03fa212cd28b5
msg370395 - (view) Author: Mark Sapiro (msapiro) * (Python triager) Date: 2020-05-30 17:38
With the fix in PR 20038, committed at https://github.com/python/cpython/commit/6f2f475d5a2cd7675dce844f3af436ba919ef92b it is no longer possible to set_content(''). Attempts to do so produce the following
```
  File "/var/MM/3/hk_39/hyperkitty/.tox/py39-django30/lib/python3.9/site-packages/django_mailman3/lib/scrub.py", line 95, in _get_all_attachments
    part.set_content('')
  File "/usr/local/lib/python3.9/email/message.py", line 1171, in set_content
    super().set_content(*args, **kw)
  File "/usr/local/lib/python3.9/email/message.py", line 1101, in set_content
    content_manager.set_content(self, *args, **kw)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 37, in set_content
    handler(msg, obj, *args, **kw)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 185, in set_text_content
    cte, payload = _encode_text(string, charset, cte, msg.policy)
  File "/usr/local/lib/python3.9/email/contentmanager.py", line 149, in _encode_text
    if max(len(x) for x in lines) <= policy.max_line_length:
ValueError: max() arg is an empty sequence
```
msg373350 - (view) Author: miss-islington (miss-islington) Date: 2020-07-08 21:00
New changeset 4fa61a7732923f92de0f7830c12da48c4cec937f by Mark Sapiro in branch 'master':
bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
https://github.com/python/cpython/commit/4fa61a7732923f92de0f7830c12da48c4cec937f
msg373353 - (view) Author: miss-islington (miss-islington) Date: 2020-07-08 21:18
New changeset c1c50345933efca42169f03d79ff4fe3d9c06bdc by Miss Islington (bot) in branch '3.8':
bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
https://github.com/python/cpython/commit/c1c50345933efca42169f03d79ff4fe3d9c06bdc
msg373354 - (view) Author: miss-islington (miss-islington) Date: 2020-07-08 21:21
New changeset e68978978f49cfc489fa0f888281ce84460818c0 by Miss Islington (bot) in branch '3.9':
bpo-40597: Allow email.contextmanager set_content() to set a null string. (GH-20542)
https://github.com/python/cpython/commit/e68978978f49cfc489fa0f888281ce84460818c0
msg373362 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2020-07-09 00:18
Closing this since the PRs for the behavior change reported by Mike and BPO-41206 is now merged too.
msg373363 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2020-07-09 00:19
Thanks Mark!
History
Date User Action Args
2020-07-09 00:19:01maxkingsetmessages: + msg373363
2020-07-09 00:18:35maxkingsetstatus: open -> closed

nosy: + maxking
messages: + msg373362

stage: patch review -> resolved
2020-07-08 21:21:05miss-islingtonsetmessages: + msg373354
2020-07-08 21:18:45miss-islingtonsetmessages: + msg373353
2020-07-08 21:01:04miss-islingtonsetpull_requests: + pull_request20554
2020-07-08 21:00:56miss-islingtonsetpull_requests: + pull_request20553
2020-07-08 21:00:43miss-islingtonsetmessages: + msg373350
2020-05-30 19:55:11msapirosetstage: resolved -> patch review
pull_requests: + pull_request19786
2020-05-30 17:38:05msapirosetnosy: + msapiro
messages: + msg370395
2020-05-18 00:58:21r.david.murraysetstage: backport needed -> resolved
2020-05-18 00:57:49r.david.murraysetmessages: + msg369157
2020-05-14 00:57:34r.david.murraysetresolution: fixed
stage: patch review -> backport needed
messages: + msg368811
versions: - Python 3.5, Python 3.6, Python 3.7
2020-05-14 00:53:40miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19388
2020-05-14 00:53:34r.david.murraysetmessages: + msg368810
2020-05-11 20:43:39r.david.murraysetmessages: + msg368646
2020-05-11 19:10:54ivylsetkeywords: + patch
stage: patch review
pull_requests: + pull_request19348
2020-05-11 19:00:19ivylcreate