This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: email.policy.SMTP and SMTPUTF8 doesn't honor linesep's value
Type: behavior Stage: resolved
Components: email Versions: Python 3.6
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: barry, maxking, msapiro, r.david.murray
Priority: normal Keywords:

Created on 2017-11-27 00:03 by maxking, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (6)
msg307029 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2017-11-27 00:03
According to the documentation, email.policy.SMTP and .SMTPUTF8  should have CRLF (`\r\n`) as line endings for text/plain parts.

However, a new message parsed using message_from_binary_file with policy.SMTPUTF8 has `\n` as line endings for text/plain parts. This happens when I use get_content() on the new EmailMessage API.

Just to mention, in Python 2.7, the old `get_payload(decode=True)` returns the body with CRLF as line endings. However, in Python 3, `get_payload()` also returns '\n' as line endings.

Am I reading the documentation wrong?
msg307089 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-11-27 21:54
Your are reading the documentation wrong.  For linesep, it says "The string to be used to terminate lines in serialized output."  The key word there is "output".  email operates in "universal newline mode" when parsing messages, turning all line endings into python's internal \n format.  What the linesep policy controls is the *output* line separator.

Note that the python2 behavior for decode=True is arguably a bug, but not one we are going to fix for backward compatibility reasons.
msg307093 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2017-11-27 22:04
I understand that the line endings are meant for "outputs", but what constitutes as output in this case?

I now see that `EmailMessage.as_string()` gives me `\r\n` as line endings and, which is probably what you mean by output and `get_content()` given me internal representation of the Email's content?
msg307119 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-11-28 13:59
OK, I should have used the term the docs actually use: "serialization".  get_content() is, I hope, clearly not serialization, while as_string() is.  Does that make it make more sense?  Do you see a way to improve the docs in this regard?
msg307276 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2017-11-30 03:16
Well, I myself missed the "serialization" part, so it probably was my mistake. Maybe a :note: saying line endings are only added in serialized outputs like the one obtained from `as_string()` or `as_bytes()` but not for other methods that return the body like `get_content()` or `get_payload()` might be helpful.

I stumbled upon it while porting some code from Python 2 to Python 3 and decided to use the new API. But, given the behavior of Python 2's get_content(decode=True), I would say it might just be clearer if docs mentioned this change in behavior (even though the new one is the "correct" one).

I can send in a pull request with modifications if you'd like.
msg307324 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-11-30 16:08
A note would be too heavy handed.  And get_content is a different function from get_payload, so there is no change in behavior, there is a new API with a more consistent behavior.  Basically, Python programs in general use \n line endings internally, and email was an exception to that rule (and is no longer, in the new API).  (There's a now closed issue about this somewhere in the tracker.)
Date User Action Args
2022-04-11 14:58:54adminsetgithub: 76325
2017-11-30 16:08:11r.david.murraysetmessages: + msg307324
2017-11-30 03:16:15maxkingsetmessages: + msg307276
2017-11-28 13:59:52r.david.murraysetmessages: + msg307119
2017-11-27 22:04:11maxkingsetmessages: + msg307093
2017-11-27 21:54:53r.david.murraysetstatus: open -> closed
resolution: not a bug
messages: + msg307089

stage: resolved
2017-11-27 05:39:56msapirosetnosy: + msapiro
2017-11-27 00:03:08maxkingcreate