It turns out that the RFC is unambiguous on this point.  Quoting the
base64 section of RFC 2045:

   The encoded output stream must be represented in lines of no more
   than 76 characters each.  All line breaks or other characters not
   found in Table 1 must be ignored by decoding software.  In base64
   data, characters other than those in Table 1, line breaks, and other
   white space probably indicate a transmission error, about which a
   warning message or even a message rejection might be appropriate
   under some circumstances.

Since "some circumstances" is not something the base64 decoder can
decide, that has to be left to a higher level ap.  So if unexpected
characters are to generate an error, it would need to be enabled via a
flag that defaults to not raising the error, IMO.  Unless someone has a
use case for rejecting an improperly encoded message, we should probably
just fix the docs (perhaps noting that this behavior is in accordance
with the RFC).