This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author ncoghlan
Recipients asvetlov, grahamd, mhammond, ncoghlan, pitrou
Date 2012-08-29.02:19:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1346206754.17.0.487622792732.issue15751@psf.upfronthosting.co.za>
In-reply-to
Content
The reason I'm proposing going back to the original SwitchInterpreter idea is because the EnsureEx idea doesn't nest cleanly - if your thread is already associated with "interpreter A", you can't readily call into "interpeter B", because the API provides no way to correctly restore the associated interpreter back to interpreter A when you're done.

EnsureEx works fine if extension modules are not aware of multiple interpreters:

   /* Embedding application (GIL always unlocked) */
   gilstate = PyGILState_EnsureEx(interp_A);
     /* Python code and extension code happens */
       /* Callback needs to reach back into Python */
       cb_gilstate = PyGILState_Ensure();
         /* Callback runs */
       PyGILState_Release(cb_gilstate);
     /* Python code and extension code unwinds */
   PyGILState_Release(gilstate);

However, it *doesn't* work (at least, not easily) if the extension itself wants to call back into an interpreter other than the one already associated with the current thread:

   /* Embedding application (GIL always unlocked) */
   gilstate = PyGILState_EnsureEx(interp_A);
     /* Python code and extension code happens */
       /* Callback needs to reach back into a specific interpreter */
       cb_gilstate = PyGILState_EnsureEx(interp_B);
         /* Callback runs */
       PyGILState_Release(cb_gilstate);
     /* Python code and extension code unwinds */
   PyGILState_Release(gilstate);

Does that second call to EnsureEx fail? If it succeeds, how does the client know which interpreter to use for the PyGILState_Release call? It could be made to work if PyGILState_STATE was changed from an enum to a struct that included in interpreter state pointer, or if EnsureEx returned a different type and was paired up with a new ReleaseEx pointer.

However, that's starting to get very messy compared to a separate SwitchInterpreter call:

   /* Embedding application (GIL always unlocked) */
   old_interp = PyGILState_SwitchInterpreter(interp_A);
   /* "autoTLSkey" now refers to a thread state for interpreter A */
   gilstate = PyGILState_Ensure();
     /* Python code and extension code happens */
       /* Callback needs to reach back into Python */
       pre_cb_interp = PyGILState_SwitchInterpreter(interp_B);
       /* "autoTLSkey" now refers to a thread state for interpreter B */
       cb_gilstate = PyGILState_Ensure();
         /* Callback runs */
       PyGILState_Release(cb_gilstate);
       PyGILState_SwitchInterpreter(pre_cb_interp);
       /* "autoTLSkey" again refers to a thread state for interpreter A */
     /* Python code and extension code unwinds */
   PyGILState_Release(gilstate);
   PyGILState_SwitchInterpreter(old_interp);

And yes, I'm pondering ways that this could be used to implement Rust-style "channels" for communication between interpreters without needing to copy data, by using this API to create proxy interfaces for accessing an object owned by another subinterpreter.
History
Date User Action Args
2012-08-29 02:19:14ncoghlansetrecipients: + ncoghlan, mhammond, pitrou, grahamd, asvetlov
2012-08-29 02:19:14ncoghlansetmessageid: <1346206754.17.0.487622792732.issue15751@psf.upfronthosting.co.za>
2012-08-29 02:19:13ncoghlanlinkissue15751 messages
2012-08-29 02:19:12ncoghlancreate