Author _savage
Recipients _savage, barry, maciej.szulik, matrixise, python-dev, r.david.murray
Date 2018-08-06.18:51:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <>
David, I tried to find the mentioned '\r\r…\n' issue but I could not find it here. However, from an initial investigation into the BytesGenerator, here is what’s happening.

Flattening the body and attachments of the EmailMessage object works, and eventually _write_headers() is called to flatten the headers which happens entry by entry ( Flattening a header entry is a recursive process over the parse tree of the entry, which builds the flattened and encoded final string by descending into the parse tree and encoding & concatenating the individual “parts” (tokens of the header entry).

Given the parse tree for a header entry like "Martín Córdoba <>" eventually results in the correct flattened string:

    '=?utf-8?q?Mart=C3=ADn_C=C3=B3rdoba?= <>\r\n'

at the bottom of the recursion for this “Mailbox” part. The recursive callstack is then:

    fold [Mailbox]
    fold [Address]
    fold [AddressList]
    fold [Header]
    fold [_UniqueAddressHeader]
    _fold [EmailPolicy]
    fold_binary [EmailPolicy]
    _write_headers [BytesGenerator]
    _write [BytesGenerator]

The problem now arises from the interplay of 

    encoded_part = part.fold(policy=policy)[:-1] # strip nl

which strips the '\n' from the returned string, and

    return policy.linesep.join(lines) + policy.linesep

which adds the policy’s line separation string linesep="\r\n" to the end of the flattened string upon unrolling the recursion.

I am not sure about a proper fix here, but considering that the linesep policy can be any string length (in this case len("\r\n") == 2) a fixed truncation of one character [:-1] seems wrong. Instead, using:

    encoded_part = part.fold(policy=policy)[:-len(policy.linesep)] # strip nl

seems to work for entries with and without Unicode characters in their display names.

David, please advise on how to proceed from here.
Date User Action Args
2018-08-06 18:51:49_savagesetrecipients: + _savage, barry, r.david.murray, python-dev, maciej.szulik, matrixise
2018-08-06 18:51:49_savagesetmessageid: <>
2018-08-06 18:51:49_savagelinkissue24218 messages
2018-08-06 18:51:49_savagecreate