classification
Title: email.header unicode fix
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: BreamoreBoy, barry, r.david.murray, tkikuchi
Priority: normal Keywords: easy, patch

Created on 2007-03-15 11:13 by tkikuchi, last changed 2011-03-17 15:23 by barry. This issue is now closed.

Files
File name Uploaded Description Edit
email_header_unicode.patch tkikuchi, 2007-03-16 02:01 email_header_unicode.patch review
Messages (7)
msg52236 - (view) Author: Tokio Kikuchi (tkikuchi) Date: 2007-03-15 11:13
email.header.Header class has its unicode representation which is slightly different from str representation.

>>> h = email.Header.make_header((('[XXX]', 'us-ascii'), ('Re:', 'us-ascii')))
>>> unicode(h)
u'[XXX]Re:'
>>> str(h)
'[XXX] Re:'

Note that a white space between ']' and 'R' is absent from unicode representation.  A word-separating space should be needed in following contexts:

lastcs \ nextcs | ascii | other |
     ascii      |  sp   |   sp  |
     other      |  sp   | nosp  |

Current code fails to put a space for ascii\ascii case.

Also, if the ascii string has a trailing space, it may result in a extraneous double space which has semantically equivalent to a single space and sometimes annoying.  You should use qp encoding for ascii charset if you insist on the presence of the space.

The patch includes test code. :)
msg52237 - (view) Author: Tokio Kikuchi (tkikuchi) Date: 2007-03-16 02:01
Sorry but I am withdrawing the latter part of the patch for ascii encoding.  Revised patch is for only unicode representation.

File Added: email_header_unicode.patch
msg110567 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-17 15:38
Applied the patches to the test files for 2.6.5 without patching header.py and got one failure.

FAIL: test_i18nheader_unicode (email.test.test_email.TestRFC2047)

I also expected a failure from the equivalent test for test_email_renamed, can someone please advise.
msg131213 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-17 03:46
I'm rejecting this.  There is more than one bug here, but it starts with the fact that ('xxx', None) is treated the same as ('xxx', 'us-ascii').  In the first case it would be nice if a space was used to separate two of them in a row.  In the second case a space should not be used, since it should in fact be an encoded word.

The documentation for make_header says it takes a list produced by decode_header.  decode_header will never produce two ('xxx', None) tuples in a row.  It could produce a tuple with us-ascii preceeded or followed by one with None, but that would be only if the us-ascii one really was an encoded word, and so encoded word spacing rules should be followed.

However, the current quirks are of such long standing that I don't think it is a good idea to fix them.  The broken behavior won't be encountered in any reasonable email, and changing the behvior is much more likely to break existing oddball uses of the interface than it is to fix bugs in such uses.

We'll fix this to work right in email6.
msg131223 - (view) Author: Tokio Kikuchi (tkikuchi) Date: 2011-03-17 05:18
OK. I understand but slightly disappointed because four years have passed since this issue was raised and these years are included in the 'such long standing' period.
msg131243 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-17 12:59
Yes, I can well understand that feeling.  I've only relatively recently taken over maintaining the email package. I'm working my way through the old bug queue, and I can only deal with them in the context in which I find them.
msg131261 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2011-03-17 15:23
Sadly, I agree with RDM.  There are too many workarounds in the field for this bug so it can really only be fixed in email6.
History
Date User Action Args
2011-03-17 15:23:15barrysetnosy: barry, tkikuchi, r.david.murray, BreamoreBoy
messages: + msg131261
2011-03-17 12:59:08r.david.murraysetnosy: barry, tkikuchi, r.david.murray, BreamoreBoy
messages: + msg131243
2011-03-17 05:18:56tkikuchisetnosy: barry, tkikuchi, r.david.murray, BreamoreBoy
messages: + msg131223
2011-03-17 03:46:28r.david.murraysetstatus: open -> closed

messages: + msg131213
resolution: rejected
nosy: barry, tkikuchi, r.david.murray, BreamoreBoy
2011-03-13 22:36:42r.david.murraysetnosy: barry, tkikuchi, r.david.murray, BreamoreBoy
versions: + Python 3.1, Python 2.7, Python 3.2, Python 3.3, - Python 2.6
2010-12-27 17:04:58r.david.murrayunlinkissue1685453 dependencies
2010-07-17 15:38:35BreamoreBoysetnosy: + BreamoreBoy
messages: + msg110567
2010-05-05 13:46:11barrysetassignee: barry -> r.david.murray

nosy: + r.david.murray
2009-04-22 05:08:27ajaksu2setkeywords: + easy
2009-03-30 22:58:29ajaksu2setstage: patch review
type: behavior
versions: + Python 2.6, - Python 2.5
2009-03-30 22:56:23ajaksu2linkissue1685453 dependencies
2007-03-15 11:13:52tkikuchicreate