Title: email.feedparser: support RFC 6532 section 3.5
Type: behavior Stage: patch review
Components: email Versions: Python 3.11
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, f18a14c09s, python-dev, r.david.murray, terry.reedy, virusdetected1337
Priority: normal Keywords: patch

Created on 2021-07-17 03:52 by f18a14c09s, last changed 2021-08-02 14:14 by f18a14c09s.

Pull Requests
URL Status Linked Edit
PR 27208 open python-dev, 2021-07-17 07:39
Messages (8)
msg397693 - (view) Author: Francis Johnson (f18a14c09s) * Date: 2021-07-17 03:52
Note that I have created a fix for this and am working through the Python Developer’s Guide to propose it.
msg398094 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-07-23 20:52
RFC 6532 Internationalized Email Headers

3.5.  Changes to MIME Message Type Encoding Restrictions

don't obviously "message/global Emails with non-identity Content-Transfer-Encodings".  Please clarify.
msg398100 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-07-23 21:44
dont obviously *match* ....
msg398101 - (view) Author: Francis Johnson (f18a14c09s) * Date: 2021-07-23 21:45
Try parsing an example such as this through the Python email library:

Content-Type: message/global
Content-Transfer-Encoding: quoted-printable

Content-Type: text/plain; =

Hello, World!

You will find that the valid quoted-printable content is visited but not decoded prior to parsing; it is parsed as-is and treated as a bad message: the charset in the “text/plain” Content-Type improperly parsed, for example.  I have a draft pull request for this, which might be easier to understand.  Let me know if you need further clarification.
msg398103 - (view) Author: blackrose 1337 (virusdetected1337) Date: 2021-07-23 23:35
#44713: subprocess.rst typo ``"shell=True"`` => ``shell=True``
msg398107 - (view) Author: Francis Johnson (f18a14c09s) * Date: 2021-07-24 00:27
The first paragraph of section 3.5 states two positions that the RFC holds on Content-Transfer-Encodings:
(1) “allows newly defined MIME types to permit content-transfer-encoding;” and
(2) “allows content-transfer-encoding for message/global (see Section 3.7) [sic].”

The second position refers to and combines with section 3.7 (namely the “Encoding considerations” paragraph) to support my interpretation that implementations should accurately parse "message/global Emails with non-identity Content-Transfer-Encodings” (how I have paraphrased it).  For quick reference, “non-identity” refers to the “quoted-printable” and “base64” Content-Transfer-Encodings.
The first position “suggests” a wider breadth, but I do not think its words “necessitate” it.  For, the RFC lists only one “newly defined MIME type;” whereas a media/MIME type in general (in this case, all others) only affects the scope of RFCs that define/update it.

So I think my interpretation is precise if one defers to section 3 of the RFC.

Now, admittedly, two other sections in the RFC seem to contradict section 3.5 by broadening the scope (Abstract, p. 2; Introduction, s. 1, p. 3).  I’m not super hung up on how to resolve the contradiction; but I think, at a minimum, the missing support for “message/global” should be added.
msg398108 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2021-07-24 00:53
Having looked at the cited part of the RFC (but not tried to analyze it in detail), I think you are correct.  I've also glanced at your PR, and I think your approach is correct in broad outline, but I haven't looked at the details.  For full message/global support, however, it will also be necessary to look at the output side: given a message/global part, a transfer encoding should be applied when serializing with cte_type=7bit.  Support for message/global should also be added to the contentmanager.

I won't have an objection if this is accepted with only the feedparser support, but I would recommend that the remaining pieces of support for message/global be added before the feature is released.
msg398773 - (view) Author: Francis Johnson (f18a14c09s) * Date: 2021-08-02 14:14
As requested, I’ve started testing and writing changes for the generator and content manager.  I’m assuming there is no rush since the scope is now widened.  Will have some follow-on questions about the design.
Date User Action Args
2021-08-02 14:14:51f18a14c09ssetmessages: + msg398773
2021-07-24 00:53:09r.david.murraysetmessages: + msg398108
2021-07-24 00:27:53f18a14c09ssetmessages: + msg398107
2021-07-23 23:35:10virusdetected1337setnosy: + virusdetected1337
messages: + msg398103
2021-07-23 21:45:36f18a14c09ssetmessages: + msg398101
2021-07-23 21:44:33terry.reedysetmessages: + msg398100
2021-07-23 20:52:57terry.reedysetnosy: + terry.reedy

messages: + msg398094
title: email.feedparser Module Lacks Support for Section 3.5 of RFC 6532: message/global Emails with non-identity Content-Transfer-Encodings -> email.feedparser: support RFC 6532 section 3.5
2021-07-17 07:39:47python-devsetkeywords: + patch
nosy: + python-dev

pull_requests: + pull_request25746
stage: patch review
2021-07-17 03:52:51f18a14c09screate