Author Eric Lafontaine
Recipients Eric Lafontaine, Henning.von.Bargen, maciej.szulik, r.david.murray
Date 2016-12-19.02:40:46
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
Hi all,

this is not a short answer, like I hoped it would be ;).  But skip to the last section if you don't want to read it all.
the heuristic problem :

For the heuristic of the resent headers, it's clearly say in the RFC5322 that all resent block should be PREPENDED to messages.  So only the first ones you see should be handled.  email.message keeps the order of the headers while reading and "get" always takes the first one if it exist :).    

from the RFC: 
   Each new set of resent fields is prepended to the
   message; that is, the most recent set of resent fields appears
   earlier in the message.  No other fields in the message are changed
   when resent fields are added.

For the behavior of the send_message, we used Resent-Date as it should always be present for EACH set of resent : 
   When resent fields are used, the "Resent-From:" and "Resent-Date:"
   fields MUST be sent.  The "Resent-Message-ID:" field SHOULD be sent.
   "Resent-Sender:" SHOULD NOT be used if "Resent-Sender:" would be
   identical to "Resent-From:".
It's fantastic that things work well like this for coders :).

The issues is with the msg object being passed...  It does retain the order, but doesn't prepend new-headers... it only append new headers (putting them at the bottom of the e-mail).

If someone wants to prepend headers (i.e. the "Resend-"), they will have to take the msg obj and do the addition manually; 
    msg._headers.insert(0,msg.policy.header_store_parse(name, val)).
They already have to do it though as the as_string function will print them at the bottom anyway.  Changing the "__setitem__" of message change the behavior too much...  even though it would be the right thing to do... email.message read e-mail from top to bottom and "set" each line.  In other words, for order dependant headers like the Resent, we're screwed.

i.e. from the test case about the multiple resent, here is what is sent down the "data" smtp command :
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
To: John
CC: Sally, Fred
Bcc: John Root <root@localhost>, "Dinsdale" <>
Resent-Date: Thu, 1 Jan 1970 17:42:00 +0000
Resent-To: Martha <>, Jeff
Resent-Date: Thu, 2 Jan 1970 17:42:00 +0000
Resent-From: Martha <>, Jeff
Date: Sun, 18 Dec 2016 20:32:11 -0000

A test message


Finally, for the "ehlo_or_helo_if_needed", I thought about it and here's what I think and what I believe we should be doing (I'm not attached to the idea though);
    The send_message is a higher level of thinking and should be able to be modified if someone wants to act differently by sub-classing the smtplib.SMTP class.  in other words, if that someone wants to modify the SMTP behavior, he could modify the sendmail function, which would also impact the send_message function.
    This leads the developpers to have to use ehlo and prepare the connection for send_message, but doesn't enforce it :).  Which would also already be the case of doing it with sendmail and that instead of having to over-ride 2 methods, only one is over-written.  
    If this is not desired, than I suggest "protecting our reference" by using thunder sendmail instead.
I prefer to remove the ehlo from the send_message and allow "flexibility" on the send_message itself.  

The issue I have right now, is what David Mentionned about the heuristic... I should raise an error when 2 Resent-Date are found, but that will prevent me from using it for production ( as I have more than one resent field present sometime...).  This would also prevent people from using it and I find it sad.  For now, i'm guessing that adding a keyword parameter "guess" would be the right things to do, but will let that be in an other ticket.  David, I find that somehow the RFC5322 isn't clear enough on the "Resent-" header order, so I've written to the IETF to have some explanation on the rules in edge cases.  The heuristic is implemented (takes the first one it finds), but just not activated yet for multiple Resent-Date.

I'll re-iterate my need for someone to review the code though :).

Eric Lafontaine
Date User Action Args
2016-12-19 02:40:53Eric Lafontainesetrecipients: + Eric Lafontaine, r.david.murray, maciej.szulik, Henning.von.Bargen
2016-12-19 02:40:53Eric Lafontainesetmessageid: <>
2016-12-19 02:40:53Eric Lafontainelinkissue28879 messages
2016-12-19 02:40:50Eric Lafontainecreate