classification
Title: error flattening complex smime signed message
Type: behavior Stage: test needed
Components: email Versions: Python 3.3
process
Status: open Resolution:
Dependencies: 1590744 1670765 Superseder:
Assigned To: Nosy List: ajaksu2, barry, gagern, hta, ludo, r.david.murray, shevek, srikanths
Priority: normal Keywords: easy, patch

Created on 2004-06-07 20:34 by ludo, last changed 2012-05-24 03:23 by r.david.murray.

Files
File name Uploaded Description Edit
email_smime_bug.zip ludo, 2004-06-07 20:34 test messages and ca certificate file
Messages (7)
msg21076 - (view) Author: Ludovico Magnocavallo (ludo) Date: 2004-06-07 20:34
Python 2.3.3 [GCC 3.2.2] on linux2
email version 2.5.5

Complex SMIME signed messages parsed and flattened
again do not pass SMIME verification. I have noticed
this with messages that have as message/rfc822
attachment another SMIME signed message.

A diff between an "original" SMIME signed messaged
passign openssl smime -verify verification and the same
message parsed (message_from_file) and flattened
(as_string(False)) by the email library:
diff -bB bugmsg_signed.eml bugmsg_signed_parsed.eml
2c2,3
< Content-Type: multipart/signed;
protocol="application/x-pkcs7-signature"; micalg=sha1;
boundary="----381546B4549948B9F93D885A82884C49"
---
> Content-Type: multipart/signed;
protocol="application/x-pkcs7-signature";
>       micalg=sha1;
boundary="----381546B4549948B9F93D885A82884C49"

The email-parsed message splits the signature header
into two lines, thus rendering the message non-valid.

Attached to this bug a .zip archive with:

- msg #1: the non-signed message (with a signed message
as attachment)
- msg #2: message #1 signed by openssl
- msg #3: message #2 parsed and flattened as described
above
- the CA certificate file used for smime verification

openssl command used to verify #2 and #3:

openssl smime -verify -in bugmsg_signed.eml -CAfile
cacert.pem

openssl smime -verify -in bugmsg_signed_parsed.eml
-CAfile cacert.pem

msg21077 - (view) Author: Bas Wijnen (shevek) Date: 2005-01-24 11:43
Logged In: YES 
user_id=42389

I would like to add that I think this bug is quite
important, as mailman uses python.  This means that many
mailing lists invalidate signatures when signed e-mails with
attachments are sent through them.  As attachments are often
code patches, it is quite important that the signatures are
working correctly.
msg21078 - (view) Author: Bas Wijnen (shevek) Date: 2005-01-25 09:37
Logged In: YES 
user_id=42389

In case it is any help, I tried to find the problem in the
source.  I don't speak python, so I can't fix anything, but
I do speak C, so it's quite readable. :-)

It seems to me that the problem is in Lib/email/Message.py,
in particular in _parseparam and _get_params_preserve.  Both
these functions call strip() on the object several times
(which seems a bit overdone anyway ;-) ), which I presume
removes all whitespace around them.

I think the whitespace should somehow be saved (not
stripping will probably break many programs, so that's not a
good idea), so it can be used again when the header is
written.  set_param should of course also fill this value,
so new params get a defined separation (although the empty
string is quite acceptable).

How this should be implemented I gladly leave to someone who
actually speaks Python. :-)
msg21079 - (view) Author: Harald Tveit Alvestrand (hta) Date: 2005-09-08 06:16
Logged In: YES 
user_id=12193

Adding my voice that the bug is important. 
This is now the chief culprit in breaking signed messages in my usage of 
signed email on the net; that "signtures are so often broken" is one 
argument people often use against using signed email. 
 
msg21080 - (view) Author: Martin von Gagern (gagern) Date: 2007-02-28 12:07
At least for the header wrapping issue, adding a new handler to the default Generator implementation should provide an easy solution. I just created patch 968430 http://tinyurl.com/2chhz6 for this.

From mailman bug report 815297 http://tinyurl.com/2a8dta I understand that there is a fix in mailman for this issue as well, but I believe this patch would be a fix any python program would want, not just mailman.

There remains an issue about leading space in headers, mentioned in that mailman problem report as well. The cause is an lstrip call in FeedParser. However removing that would probably break a lot of existing programs. And modifying all methods accessing the headers to remove or add leading space would be somewhat ugly. Right now I'm trying to figure out how to save the unmodified body for every multipart/signed message, so that it can be stored with the message and used without any modifications during generation.
msg82054 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-02-14 13:56
Patch at #1670765.
msg161478 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-05-24 03:23
I think if we can fix issue 1590744, this will be completely fixed.
History
Date User Action Args
2012-05-24 03:23:30r.david.murraysetversions: - Python 2.7, Python 3.2
2012-05-24 03:23:12r.david.murraysetcomponents: + email, - Library (Lib)
2012-05-24 03:23:03r.david.murraysetassignee: r.david.murray ->
dependencies: + mail message parsing glitch
messages: + msg161478
2011-07-14 12:29:41srikanthssetnosy: + srikanths
2011-03-12 03:51:05r.david.murraysetnosy: barry, ludo, shevek, hta, gagern, ajaksu2, r.david.murray
versions: + Python 2.7, Python 3.2, Python 3.3
2010-05-05 13:44:13barrysetassignee: barry -> r.david.murray

nosy: + r.david.murray
2009-04-22 15:21:26ajaksu2setkeywords: + patch, easy
2009-02-14 13:56:42ajaksu2setnosy: + ajaksu2
dependencies: + email.Generator: no header wrapping for multipart/signed
type: behavior
messages: + msg82054
stage: test needed
2004-06-07 20:34:58ludocreate