While you are correct that latin1 may be common in this situation, I think it may still be better to have utf-8 be the default, since that is the (still emerging? :) standard.  However, you are correct to call for examples: if in the *majority* of the real-world cases it turns out latin1 is what is used, then we could default to that (or not have a default, but instead document our observations).

I don't know how we accumulate enough information to make that decision, though.  Maybe we could look at what other mail programs do?  Thunderbird, etc?  David, which mail program(s) did you use that were able to successfully send that password?

And yes, by binary passwords I mean that the module needs to support being passed a bytes-like object as the password, since clearly there are servers "in the wild" that support non-ascii passwords and the only way to be sure one can send the server the correct password is by treating it as a series of bytes.  The library caller will have to be responsible for picking the correct encoding based on local knowledge.