Regards.

The email.Generator module does not separates headers
with "\r\n".


Manlio Perillo

Logged In: YES 
user_id=12800

Correct; this is by design.  If you're worried about
protocols such as RFC 2821 requiring \r\n line endings,
don't. The smtplib module automatically ensures proper line
endings for the on-the-wire communication.

Logged In: YES 
user_id=1054957

Ok, thanks.
But what if I don't use the smtplib module?

I discovered the bug because I have written a small NNTP
server with twisted, using email module for parsing...

Logged In: YES 
user_id=12800

The module that speaks the wire protocol should do the
conversion.  IMO, there's no other way to guarantee that
you're RFC compliant.  You could be getting your data from
the email package, but you could be getting it from anywhere
else, and /that/ source may not be RFC line ended either. 
Since you can't change every possible source of data for
NNTP or SMTP, your network interface must guarantee conformance.

Logged In: YES 
user_id=1054957

I do not agree here (but I'm not an expert).

First - the documentation says:
"""The email package attempts to be as RFC-compliant as
possible, supporting in addition to RFC 2822, such
MIME-related RFCs as RFC 2045, RFC 2046, RFC 2047, and RFC 2231.
"""

But, as I can see, the generated email does not conform to
RFC 2822.

Second - I use email package as a "filter".
read raw email text, do some processing, generate raw email
text.

Really, I don't understand why generated headers don't are
separed by '\r\n' and one must rely on an external tool for
the right conversion.

Thanks.

Logged In: YES 
user_id=12800

I hear what you're saying, but so far, it has been more
convenient for developers when the generator outputs native
line endings.  I can see a case for a flag or other switch
on the Generator instance to force RFC 2822 line endings.

I would suggest joining the email-sig and posting a request
there so the issue can be discussed as an RFE.

Logged In: YES 
user_id=1054957

But the generator does not output in native line endings!

On Windows:
>>> from email.Message import Message
>>> msg = Message()
>>> msg["From"] = "me"
>>> msg["To"] = "you"
>>> print repr(msg.as_string())
'From: me\nTo: you\n\n'

Hi,

could you please reconsider closing this bug and consider fixing it or at least providing an option for standard behaviour?
Leaving aside the question of performance impact of postprocessing in longer mails (for those, email may not a be good optionin the first place), the module as is renders the email.Generator mostly useless for multipart messages with binary data that needs to be standards compliant, e.g. Multipart-Messages containing images, possibly signed or uploading (with httplib) multipart/form-data.

Thank you for your consideration.

Kind regards

Thomas

I am reopening this as a feature request.  I still think it's better for protocols that require these line endings to ensure that their data is standards compliant, but I can see that there may be other use cases where you'd want to generate protocol required line endings.  I'm not totally convinced, but it's worth opening the issue for now and discussing this on the email-sig.

Consider also issue 975330.

Echoing the comment of Thomas Viehmann, the current behaviour makes it impossible to use this library to generate a correct multipart/mixed message with 7_or_8_bit encoding on Unix.

The MIME standard specifies that headers are to be CRLF terminated - this is independent of any lower-level transport encoding.  

Binary bodies are simply octet sequences - so may contain \n or \r characters.

The correct MIME encoding would have the headers terminated with CRLF, and the bodies as raw byte sequences.  However on Unix, since the headers are generated with \n not CRLF, you can't even post-process this (e.g. message.as_string().replace('\n', '\r\n') as that will screw up the binary body data.

The current behaviour is basically wrong and should be fixed.

Malcolm: a Content-Transfer-Encoding of 8bit may only contain \r and \n characters as part of the line ending sequence.  8bit is *not* binary; to use a CTE of binary the SMTP server must support BINARYMIME, which I don't think is all that common yet.  At any rate, my up-to-date postfix server doesn't support it.

And in any case, the email package doesn't support the binary CTE, so it's kind of irrelevant anyway :)

All that said, however, it certainly seems useful to be able to generate crlf terminated messages as an option.  And it turns out that that capability is needed in order to be able to generate binary messages in Python3 while still remaining backward compatible with the behavior of Python2 when parsing binary messages (see issue 10134).  (Actually I think the scenario I'm finding problematic in Python3 is also problematic in Python2, it's just that there are tricks you can use to work around it in Python2 that aren't allowed in Python3 because of the byte/string separation.)

So, attached find a patch implementing a linesep option in Generator.flatten and Header.encode.

Note that this patch also fleshes out the currently abbreviated documentation for BytesGenerator.

David: Great to see a patch for this.

You're right of course, 8bit isn't binary - I meant binary.  The main place this shows up is when you're using MIME not in email (e.g. on the web), where binary transport is entirely possible.

This fix should mean that the MIME libraries are much more usable in non-email environments.  Thanks.

Updated patch that adds a missing test for BytesGenerator.flatten and fixes the bugs in it.  Also added versionchanged tags to the docs for the linesep argument.

I think this is ready to go in.

Removed some debugging cruft from the latest patch.

Committed in r85811.