classification
Title: MIME renderer: wrong header line break with long subject?
Type: Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: barry Nosy List: aalbrecht, barry, cjw296, ggenellina, kayne, kxroberto
Priority: high Keywords:

Created on 2007-01-26 10:04 by kxroberto, last changed 2009-03-30 22:44 by barry. This issue is now closed.

Messages (10)
msg31096 - (view) Author: kxroberto (kxroberto) Date: 2007-01-26 10:04
>>> from email.MIMEText import MIMEText
>>> o=MIMEText('hello')
>>> o['Subject']='1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4
5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 '
>>> o.as_string()
'Content-Type: text/plain; charset="us-ascii"\nMIME-Version: 1.0\nContent-Transf
er-Encoding: 7bit\nSubject: 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8
9 1 2 3 4 5 6 7 8\n\t9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 \n\
nhello'
>>>


The '6 7 8\n\t9 1 2 3'  clashes together to 6 7 89 1 2 3
without space between 89 in usual mail readers.
Is this an error and should be :

'6 7 8 \n\t9 1 2 3'


? 

as there is also the space preserved in 
'6 7 8 9 \n\nhello'
msg31097 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2007-03-11 21:16
Tokio, I'm not so sure about adding that extra space.  I tested all the mail readers at my disposal on Linux and OSX.  I don't have any Windows machines available so I couldn't test Outlook or OE.  (If you have them, please indicate  what they do in a comment here!)  Here's what I found:

OSX:
  Mail.app 2.1.1 -- the extra space produces an extra space between the 8 and 9 both in the message summary and in the Subject header in the preview pane.  The no extra space message looks fine.
  Thunderbird 1.5.0.10 -- The extra space message produces an extra space in the message summary, while the no extra space message looks fine here.  But the weird thing is that in both the extra space and non-extra space cases, the Subject header in the preview pane both have extra spaces.  It looks to me like the leading tab is being inserted into the Subject header view.
  Entourage 11.3.3 -- The extra space shows up in both the summary and preview panes but only in the message with the extra space

Linux
  Thunderbird 1.5.0.10 -- both messages get a tab between the 8 and 9 in both the summary and preview pane.  This is for both the extra space message and the no-extra space message; afaict, there's no difference between the two and this is definitely a difference between the Linux version and the OSX version of the same mail reader.
  Evolution 2.9.92 -- the extra space message produces an extra space between the 8 and 9 in both the summary and preview panes.  The no-extra space message looks fine.
  Sylpheed Claws 2.6.0 -- the extra space message produces an extra space between the 8 and 9 in both the summary and preview panes.  The no-extra space message looks file.

So anyway, afaict, the extra space is not appropriate for any of the non-Windows mail readers.
msg31098 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2007-03-11 21:17
Whoops, this wasn't supposed to be a response to Tokio, but instead the OP.
msg31099 - (view) Author: kxroberto (kxroberto) Date: 2007-06-08 12:11
What would be the RFC definition for a correct auto-line break in a (subject) mail header line?
Wouldn't it be more simple to not do any auto-line break for the subject? or is there a requirement for the line break in the RFC. Don't think any reader program would fail because of >79 char subject header lines.
msg31100 - (view) Author: Gabriel Genellina (ggenellina) Date: 2007-06-09 05:19
Quoting RFC2822 section 2.2.3 <ftp://ftp.rfc-editor.org/in-notes/rfc2822.txt>: 

"""The general rule is that wherever this standard allows for folding white space (not simply WSP characters), a CRLF may be inserted before any WSP. For example, the header field:

           Subject: This is a test

can be represented as:

           Subject: This
            is a test

[...]Unfolding is accomplished by simply removing any CRLF that is immediately followed by WSP."""

That is, folding is done by inserting ONLY the sequence CRLF before any folding whitespace. When the header is interpreted, the whitespace is NOT removed, only the CRLF. The posted Subject header should become "Subject: 1 2 3...7 8\n\r 9 1 2...'

I think this is a bug in the email.header.Header class; its __init__ says, about the continuation_ws argument: "[it] must be RFC 2822 compliant folding whitespace (usually either a space or a hard tab) which will be prepended to continuation lines.". Folding does not involve *prepending* any whitespace, just inserting CRLF right before *existing* whitespace.

Note that this is wrong even for the old RFC 822 (with slightly different rules for line folding.)
msg31101 - (view) Author: kxroberto (kxroberto) Date: 2007-08-06 17:18
the bug appears to be quite frequent now (once one knows it :-) ). Possibly for the same reason one sees this kind of subject alteration by one space often on the usenet.
msg31102 - (view) Author: Andi Albrecht (aalbrecht) * Date: 2007-08-17 08:18
The problem occurs, when you are using strings, not when you use
the Header class directly.
The reason for this behaviour is the way the Generator creates
Header instances from strings: It initializes the class with
continuation_ws set to '\t' instead of the default ' ' 
(email/generator.py in method _write_headers).

>>> from email.MIMEText import MIMEText
>>> msg = MIMEText("Just a test.")
>>> msg["Subject"] = "foo "*22
>>> print msg.as_string()
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo
        foo foo foo foo foo 

Just a test.
>>> 

Tab as the whitespace character is no problem for most email clients. But
i.e. Outlook removes it.

When using the Header class you get the following

>>> from email.MIMEText import MIMEText
>>> from email.header import Header
>>> msg = MIMEText("Just a test.")
>>> msg["Subject"] = Header("foo "*22)
>>> print msg.as_string()
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo foo
 foo foo foo 

Just a test.
>>> 

The last message shows up in Outlook correctly.
msg66699 - (view) Author: Kayne Naughton (kayne) Date: 2008-05-12 02:07
Just struck this myself, found Andi's solution to work.

Constructing the header using email.header stops it from breaking the
line awkwardly (vs. just storing a string).

Suggest the documentation example page be updated to use header() in
place of straight strings.

(I can only guess as to they why of this, it seems not to hang the
indent more than a space when using header).
msg66700 - (view) Author: Kayne Naughton (kayne) Date: 2008-05-12 02:10
Argg, yes, as Andi explained it's the tab (not sure how I missed that on
first reading).
msg84604 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-03-30 18:28
This bug is a combination of [Issue1974] and [Issue5612].

[Issue1974] is being worked on.

[Issue5612] is fixed in Python 3, but will need more work to be fixed in
Python 2, *if* anyone cares about it...

Please followup on one of these issues...
History
Date User Action Args
2009-03-30 22:44:26barrysetstatus: open -> closed
assignee: cjw296 -> barry
2009-03-30 18:28:27cjw296setnosy: + cjw296
messages: + msg84604

assignee: barry -> cjw296
resolution: duplicate
2008-05-12 02:10:17kaynesetmessages: + msg66700
2008-05-12 02:07:52kaynesetnosy: + kayne
messages: + msg66699
2007-01-26 10:04:06kxrobertocreate