This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: smtpd strips final carraige return from received message body
Type: behavior Stage: resolved
Components: email, Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Deli Zhang, barry, r.david.murray
Priority: normal Keywords:

Created on 2015-11-05 06:51 by Deli Zhang, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
test.eml Deli Zhang, 2015-11-05 06:51 sample mail causing problem
smtplib.py Deli Zhang, 2015-11-05 07:05 smtplib.py changes base on python 2.7.10
Messages (6)
msg254086 - (view) Author: Deli Zhang (Deli Zhang) * Date: 2015-11-05 06:51
It's well known that in quoted-printable encoding the "CRLF" can be encoded to "=CRLF".
For example, in attachment file test.eml, the last line is "test message.=CRLF"
But after the mail is sent via SMTP and received by mail server (e.g. postfix), the last line will become "test message.=", 
then after decoding, you will see the unnecessary char "=" shown following "test message.".

The problem is caused by below code:
------------------------------------------------
class SMTP:
    ...
    def data(self, msg):
        ...
            q = quotedata(msg)
            if q[-2:] != CRLF:
                q = q + CRLF
            q = q + "." + CRLF
            self.send(q)
        ...
------------------------------------------------
Before it sends the message q, it will try to append the end-of-data sequence "<CRLF>.<CRLF>". 
As the implement, it checks whether there is "<CRLF>" in the message end, if yes then just need to append ".<CRLF>".
It looks rigorous and efficient, but it does not consider how mail server handle it.
As we know mail server will remove end-of-data sequence directly, and the left message is treat as mail data.

Thus the corresponding action should be taken on SMTP client side,
it's to say no need to check and just append end-of-data sequence here:
------------------------------------------------
class SMTP:
    ...
    def data(self, msg):
        ...
            q = quotedata(msg)
            q = q + CRLF + "." + CRLF
            self.send(q)
        ...
------------------------------------------------
msg254087 - (view) Author: Deli Zhang (Deli Zhang) * Date: 2015-11-05 07:05
Correct the action of appending the end-of-data sequence "<CRLF>.<CRLF>"
msg254127 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-05 17:14
RFC 2812 says:

  Note that the first <CRLF> of this terminating sequence is also the <CRLF> that ends the final line of the data (message text)

So, smtplib is correct.  If you have a server that is not respecting this, then that server is out of compliance and there isn't anything we can do about it.

However, I don't think that is your problem.  = at the end of a line actually represents a "soft carriage return", which means one that is *eliminated* in the decoded output.  If you will read section 6.7 of rfc 2045, specifically notes (2) and (3) in the second block of numbered paragraphs, you will see that an 'ultimate' = (an = at the end of an encoded block, with or without a CRLF after it), such as you have in your sample, is illegal.  Further, the recommended recovery action if one is seen while decoding is to leave the = in the decoded output, just as you are observing happening.

So, there is no bug here except in your message :)
msg254185 - (view) Author: Deli Zhang (Deli Zhang) * Date: 2015-11-06 12:06
I can understand you, while could you please consider below fact:
Once our SMTP server module smtpd.py receives the sample mail, it will remove the end-of-data sequence, that makes the "=" become the last char of mail data. I think it's inconsistent to our SMTP client module smtplib.py.

If we can just add "<CRLF>.<CRLF>" following mail data like postfix, which is influential amd authoritative in SMTP field, that will make things simple and will not make any trouble in reality situation.

I just advise this, if you think no need then I can compromise.
Thanks.
msg254194 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-06 13:44
That does indeed appear to be a bug in smtpd.py.

Even if postfix's client mode does do an unconditional add of an extra newline, it is wrong according to the RFC, so I don't think that we'd change smtplib.  Especially since we've had no other reports of the current code causing problems (the only problem case you have reported is against smtpd.py, which is a bug in smtpd.py).
msg309747 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-01-10 00:51
I'm closing this as won't fix since smtpd.py is deprecated and likely won't see much future improvements.  Please take a look at aiosmtpd as a much better third party replacement.
History
Date User Action Args
2022-04-11 14:58:23adminsetgithub: 69739
2018-01-10 00:51:28barrysetstatus: open -> closed
resolution: wont fix
messages: + msg309747

stage: needs patch -> resolved
2015-11-06 13:44:38r.david.murraysetnosy: + barry
components: + email
2015-11-06 13:44:21r.david.murraysetstatus: closed -> open
versions: + Python 3.5, Python 3.6
title: SMTP.data(msg) function may cause the last CRLF of msg lost if msg is quoted-printable encoding. -> smtpd strips final carraige return from received message body
messages: + msg254194

resolution: not a bug -> (no value)
stage: resolved -> needs patch
2015-11-06 12:06:37Deli Zhangsetmessages: + msg254185
2015-11-05 17:14:13r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg254127

resolution: not a bug
stage: resolved
2015-11-05 07:05:08Deli Zhangsetfiles: + smtplib.py

messages: + msg254087
2015-11-05 06:51:55Deli Zhangcreate