According to your results, you will save 0.94 us per call of b16decode, but loss 126 us at import time. You will get a benefit if b16decode is used more than 130 time.

Currently, if the performance is critical, the user can use binascii.unhexlify() directly or use bytes.fromhex(). base64.b16decode() is only needed if you need to ensure that the input strictly conforms to RFC 3548.

Other options, which will maximize the performance while keeping the validation is to add an option to binascii.unhexlify() for making it more strict. At the time of writing the base64 module its performance was not considered critical, and additional checks and preprocessing was implemented in Python.