This patch is related to and depends on issue 21725. I put it here for review, it can be applied (maybe with small changes) as soon as issue 21725 is fixed.

After blindly raising errors for 'SMTPUTF8' without 'BODY=.*' I found out that 'SMTPUTF8' alone is actually perfectly fine (for using international names in addresses but sending ASCII data). Which means I did a mistake in the smtpd patch :(
The attached 3rd version of the smtplib patch implements your suggestions, including re-factoring of the mail option parsing. One test is commented out until smtpd is fixed. I'll do that as -part of- issue 21795 tomorrow.

Coming back to this, I noticed a number of things wrong with the logic that I'm not sure how I missed before.  I had to rewrite pretty much all of the logic.  The good news is the patch is now smaller :)

The new tests are failing, because they are wrong: the msg argument to sendmail/mail *must* be a byte string if smtputf8 is in effect, since an email message may contain multiple encodings as well as other binary data.  (Well, the msg argument could be an ascii string, but what would be the point?)

I'm uploading this without fixing the tests because I'm out of time for now and may not get back to it before the weekend.  Perhaps someone else will have time to work on the tests before then.

New changeset 6b0e4c87bf9e by R David Murray in branch 'default':
#22027: Add RFC6531 support to smtplib.
https://hg.python.org/cpython/rev/6b0e4c87bf9e

Rewrote the tests to keep all the existing tests working against an unmodified SimSMTPServer, so that we know backward compatibility is maintained.  And as I said, I changed the tests to reflect that a non-ascii body must be provided as binary data, which means I deleted two tests.

(Note: in thinking about this, I vaguely remember some conversations where we thought it might make sense that if someone provided a message string that when encoded to utf-8 then conforms to the RFC, it would be OK to auto-encode it as utf-8, but I'm just not comfortable with that idea.  It provides insufficient value compared to the danger of blindly encoding something that is not RFC conformant...if someone does build a such a unicode message, they should have to convert it themselves before passing it to smtplib, as a sign they know what they are doing :)

Thanks for your work Milan.