First off, the server_address=('localhost', port) case: this feature is fundamentally broken without support for multiple sockets, because the server can listen on at most one address, and any single choice will often be inconsistent with clients' expectations.

For the (ip, port) case, the only question is whether an IPv4 address should create an AF_INET or dualstack (IPV6_V6ONLY=0) AF_INET6 socket; the latter needs a.b.c.d -> ::ffff:a.b.c.d translation.

Then there's the typical ('', port) case.  This should ideally create a server that accepts a connection on any supported family:
- If the system supports dualstack AF_INET6 sockets, use one.
- If the system only supports AF_INET sockets, use AF_INET.
- If the system only supports AF_INET6 sockets, use AF_INET6.
- If the system supports AF_INET and AF_INET6, but not dualstack, then you need multiple sockets.  Or go with AF_INET and hope nobody needs IPv6 on such a system.

(The legacy-free simple approach is to hard-code AF_INET6 and let the OS decide whether the socket should be dualstack or IPv6-only, but that only supports 2/4 of the above cases.)

Users of "conn, addr = self.get_request()" often expect addr to be an IPv4 2-tuple, especially when handling IPv4 requests.  So you have to normalize ('::ffff:192.0.2.1', 1234, 0, 0) to ('192.0.2.1', 1234).  For real IPv6 requests, it's often more convenient to keep returning 2-tuples ('2001:db8::1', 1234) instead of 4-tuples.

Another thing to watch out for is this HTTPServer code:
    host, port = self.socket.getsockname()[:2]
    self.server_name = socket.getfqdn(host)

getfqdn() has a special case for '0.0.0.0', which doesn't recognize the IPv6 equivalents:
    >>> import socket
    >>> socket.getfqdn('0.0.0.0')
   'redacted.corp.google.com'
    >>> socket.getfqdn('::')
   '::'
   >>> socket.getfqdn('::ffff:0.0.0.0')
   '::ffff:0.0.0.0'