Message169329
The reason I'm proposing going back to the original SwitchInterpreter idea is because the EnsureEx idea doesn't nest cleanly - if your thread is already associated with "interpreter A", you can't readily call into "interpeter B", because the API provides no way to correctly restore the associated interpreter back to interpreter A when you're done.
EnsureEx works fine if extension modules are not aware of multiple interpreters:
/* Embedding application (GIL always unlocked) */
gilstate = PyGILState_EnsureEx(interp_A);
/* Python code and extension code happens */
/* Callback needs to reach back into Python */
cb_gilstate = PyGILState_Ensure();
/* Callback runs */
PyGILState_Release(cb_gilstate);
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
However, it *doesn't* work (at least, not easily) if the extension itself wants to call back into an interpreter other than the one already associated with the current thread:
/* Embedding application (GIL always unlocked) */
gilstate = PyGILState_EnsureEx(interp_A);
/* Python code and extension code happens */
/* Callback needs to reach back into a specific interpreter */
cb_gilstate = PyGILState_EnsureEx(interp_B);
/* Callback runs */
PyGILState_Release(cb_gilstate);
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
Does that second call to EnsureEx fail? If it succeeds, how does the client know which interpreter to use for the PyGILState_Release call? It could be made to work if PyGILState_STATE was changed from an enum to a struct that included in interpreter state pointer, or if EnsureEx returned a different type and was paired up with a new ReleaseEx pointer.
However, that's starting to get very messy compared to a separate SwitchInterpreter call:
/* Embedding application (GIL always unlocked) */
old_interp = PyGILState_SwitchInterpreter(interp_A);
/* "autoTLSkey" now refers to a thread state for interpreter A */
gilstate = PyGILState_Ensure();
/* Python code and extension code happens */
/* Callback needs to reach back into Python */
pre_cb_interp = PyGILState_SwitchInterpreter(interp_B);
/* "autoTLSkey" now refers to a thread state for interpreter B */
cb_gilstate = PyGILState_Ensure();
/* Callback runs */
PyGILState_Release(cb_gilstate);
PyGILState_SwitchInterpreter(pre_cb_interp);
/* "autoTLSkey" again refers to a thread state for interpreter A */
/* Python code and extension code unwinds */
PyGILState_Release(gilstate);
PyGILState_SwitchInterpreter(old_interp);
And yes, I'm pondering ways that this could be used to implement Rust-style "channels" for communication between interpreters without needing to copy data, by using this API to create proxy interfaces for accessing an object owned by another subinterpreter. |
|
Date |
User |
Action |
Args |
2012-08-29 02:19:14 | ncoghlan | set | recipients:
+ ncoghlan, mhammond, pitrou, grahamd, asvetlov |
2012-08-29 02:19:14 | ncoghlan | set | messageid: <1346206754.17.0.487622792732.issue15751@psf.upfronthosting.co.za> |
2012-08-29 02:19:13 | ncoghlan | link | issue15751 messages |
2012-08-29 02:19:12 | ncoghlan | create | |
|