New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add option to disallow > 1 instance of an extension module #84780
Comments
When a C extension module is created with PyModuleDef_Init(), it becomes possible to create more than one instance of the module. It would take significant effort to modify some extensions to make their code fully ready to have two isolated module. For example, the atexit module calls _Py_PyAtExit() to register itself into the PyInterpreterState. If the module is created more than once, the most recently created module wins, and calls registered on other atexit instances are ignore: see bpo-40288. One simple option would be to simply disallow loading the module more than once per interpreter. Also, some extensions are not fully compatible with subinterpreters. It may be interesting to allow to load them in a subinterpreter if it's not already loaded in another interpreter, like another subinterpreter or the main interpreter. It would be only load it once per Python *process*. For example, numpy would be a good candidate for such option. I'm not sure fow a module should announced in its definition that it should not be loaded more than once. |
Title clarified. Leaving subinterpreters aside, only one instance, AFAIK, is true for stdlib and python modules unless imported with different names, as can happen with main module (which is a nuisance if not a bug). So only once per interpreter seems like a bugfix. Only once per process, if once per interpreter otherwise becomes normal, seems like an enhancement, even if a necessary option. |
What about a new PyModuleDef_Slot function?
|
One of my opinions is that Since module object supports detecting error during Py_mod_exec, we should define new exception type and don't have to define new slots. but the cons of this method is that we should promise the standard exception when try to create multiple instances which is not allowed. ref: cpython/Objects/moduleobject.c Line 399 in 788b79f
On the other side, defining a Py_mod_exec_once that supports execution for just once can be a way. Please point out what I missed :) |
One option is to get the behavior before multi-phase initialization. We store extensions in a list. Once it's load, it cannot be unloaded before we exit Python. See _PyState_AddModule() and _PyState_AddModule(). Calling PyInit_xxx() the second time would simply return the existing module object. When we exit Python, the clear and/or free function of the module is called. |
Something like this?
Then add a flag to PyModuleDef to indicate it is already exec? |
IMHO, |
|
Is it really necessary to add a slot/flag for this? static int loaded = 0;
static int
exec_module(PyObject* module)
{
if (loaded) {
PyErr_SetString(PyExc_ImportError,
"cannot load module more than once per process");
return -1;
}
loaded = 1;
// ... rest of initialization
} |
I would like to limit an extension to once instance *per interpreter*. See the atexit module for an example. |
Are there any other examples? In my view, atexit is very special, and very closely tied to interpreter. I don't think it's good to design general API for one module. |
I'm not 100% sure that preventing to create multiple module instances is needed. For the atexit module, another solution is to move the "module state" into the interpreter, as it has been done for other modules like _warnings. |
I am +1 on this solution if this module is a very special case. |
OK, I am agree victor's solution too. It's a more simpler way. |
Thanks! Indeed, that's an even better solution than I had in mind. I will close this issue and PRs. |
The atexit module was my main motivation for this issue. So I'm fine with closing it. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: