PEP 489 (Multi-phase extension module initialization) makes it possible/easy to unload/reimport extension modules, in the sense of destroying/recreating the module object. The problem is that the modules needs to opt-in to supporting this, which is not always easy (e.g. the module needs to not use C globals, or use them carefully), and sometimes it's still nearly impossible (see the in-progress PEP 573).

Unloading the actual shared library is another matter, though. That's not currently planned. There's no good way to ensure that there no remaining objects that could reference the shared library's code.
Instead, your best bet is probably to name the new .so/DLL differently, and load an extra copy. (PEP 489 makes it possible to make the .so/DLL contain a module with a different name.) If you do go this way, I'd welcome feedback.