The matter of unloading extension modules is partly covered in bpo-401713.  However, note that a few things have changed in the last 18 years. :)  I think it would be worth revisiting the decision in that issue at this point.

If that were sorted out would there be other issues to address?

Regardless, If I understood right, your only objective here is to completely reset Python in the current process to the initial state.  You mention "embedded python interpreter", but it sounds more like you mean "embedded python runtime" (particularly since you are using Py_Initialize and Py_Finalize).  Why is completely resetting Python "a strong requirement"?