This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: imaplib: must not replace LF or CR by CRLF in literals
Type: behavior Stage: patch review
Components: email, Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, cheryl.sabella, mcepl, mdk, memeplex, r.david.murray, rajeshsr, ron.duplain
Priority: normal Keywords: easy, patch

Created on 2009-03-06 07:51 by memeplex, last changed 2022-04-11 14:56 by admin.

Files
File name Uploaded Description Edit
test_imaplib_cr_lf.diff ron.duplain, 2009-04-26 20:21 an attempt at testing IMAP4.append for CR, LF preservation review
Pull Requests
URL Status Linked Edit
PR 10901 open hobbestigrou, 2018-12-04 17:12
Messages (8)
msg83241 - (view) Author: Memeplex (memeplex) Date: 2009-03-06 07:51
For example, after that "normalization", quoted printable encoded
headers (as described at rfc 2047) longer than 76 characters are
splitted in two different ill-formed headers because the soft LF line
break becomes a "hard" CRLF one. This is clearly wrong.

rfc 2060 specifically allows CR and LF inside literals:

"""
A literal is a sequence of zero or more octets (including CR and LF),
prefix-quoted with an octet count in the form of an open brace ("{"),
the number of octets, close brace ("}"), and CRLF.
"""
msg86572 - (view) Author: Ron DuPlain (ron.duplain) * (Python committer) Date: 2009-04-25 23:38
It looks like the IMAP4.append method is responsible for the CRLF
substitution (trunk/Lib/imaplib.py).

# defined near top of module:
MapCRLF = re.compile(r'\r\n|\r|\n')

# in append method:
self.literal = MapCRLF.sub(CRLF, message)

I'll work on a test for it this evening.

-Ron
msg86602 - (view) Author: Ron DuPlain (ron.duplain) * (Python committer) Date: 2009-04-26 20:21
Module imaplib has pretty sparse test code.  There is only 1 test case,
for imaplib.Time2Internaldate.
trunk/Lib/test/test_imaplib.py

The attached patch tests for LF, CR preservation with regard to the
IMAP4.append method, but more testing is necessary to make sure LF, CR
are preserved in IMAP literals throughout imaplib and that the
respective fix doesn't break anything.  Mock IMAP interaction may be the
way to go for better test cases.  Is anyone working on expanding imaplib
test coverage?

-Ron
msg94578 - (view) Author: Rajesh S R (rajeshsr) Date: 2009-10-27 20:32
Am quite new here; just searching hard to contribute, would like to
patch this, if I can go ahead.

Don't we need to patch the original imaplib code also?

just remove the line:
self.literal = MapCRLF.sub(CRLF, message)
and have:
self.literal = message
Or am I missing something completely?
msg315575 - (view) Author: Matej Cepl (mcepl) * Date: 2018-04-21 18:33
> Module imaplib has pretty sparse test code.

On that note: would anybody be willing to express in form of (simplified) test case, what's the problem reported here? I am not sure, I follow.
msg315577 - (view) Author: Matej Cepl (mcepl) * Date: 2018-04-21 18:53
Oh, this is 2.6 bug. This should be closed.
msg316170 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-05-04 13:02
I'm not sure if this is a 2.6/2.7 only issue because the code mentioned in msg86572 still exists, although with a change for #21800 (RFC 6855/UTF-8) applied.

For reference, #25591 expanded the test coverage for the imaplib.  However, I don't see any tests related to CR or LF in literals.
msg352292 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2019-09-13 11:38
It looks like a revert of 47404fffff3e36699786082d0ee6565872d627e1
Which is the fix for https://bugs.python.org/issue723962 which I'm currently reading.
History
Date User Action Args
2022-04-11 14:56:46adminsetgithub: 49680
2020-11-07 01:52:35iritkatrielsetversions: + Python 3.8, Python 3.9, Python 3.10, - Python 2.6, Python 2.7
2019-09-13 11:38:50mdksetnosy: + mdk
messages: + msg352292
2018-12-04 17:12:10hobbestigrousetstage: test needed -> patch review
pull_requests: + pull_request10141
2018-05-04 13:02:34cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg316170
2018-04-21 18:53:37mceplsetmessages: + msg315577
2018-04-21 18:33:35mceplsetmessages: + msg315575
2017-10-23 16:03:20mceplsetnosy: + mcepl
2013-01-23 15:03:21r.david.murraysetnosy: + barry, r.david.murray
components: + email
2009-10-27 20:32:57rajeshsrsetnosy: + rajeshsr
messages: + msg94578
2009-04-26 20:21:12ron.duplainsetfiles: + test_imaplib_cr_lf.diff
keywords: + patch
messages: + msg86602

versions: + Python 2.7
2009-04-25 23:38:47ron.duplainsetnosy: + ron.duplain
messages: + msg86572
2009-04-22 14:38:28ajaksu2setkeywords: + easy
stage: test needed
2009-03-06 07:52:05memeplexsettitle: Must not replace LF or CR by CRLF in literals -> imaplib: must not replace LF or CR by CRLF in literals
2009-03-06 07:51:15memeplexcreate