classification
Title: PyGILState_Ensure does not acquires GIL
Type: enhancement Stage:
Components: Documentation Versions: Python 3.2
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: docs@python Nosy List: amaury.forgeotdarc, docs@python, georg.brandl, glchapman21, kunoospald, ncoghlan, pitrou, terry.reedy, urBan_dK
Priority: normal Keywords:

Created on 2007-05-16 19:46 by kunoospald, last changed 2011-02-28 14:48 by pitrou. This issue is now closed.

Messages (7)
msg32027 - (view) Author: Kuno Ospald (kunoospald) Date: 2007-05-16 19:46
The function PyGILState_Ensure doesn't acquire the GIL if current thread state is valid. In contrast to that PyGILState_Release deletes the thread state (PyThreadState_DeleteCurrent) which releases the GIL which got not acquired before (=> mutex->owned = -2).

Here is an example which locks at PyRun_SimpleString:

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

// release the GIL as PyEval_InitThreads 
// implicitly acquires the GIL
PyEval_ReleaseLock();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate); 

In that simple example the problem can be fixed by removing the call to PyEval_ReleaseLock. But that is needed for applications that call into the interpreter from multiple threads. 

The only solution I could found up to that point is the following:

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

PyThreadState* tcur = PyThreadState_Get() ;

PyThreadState_Swap(NULL);
PyThreadState_Clear(tcur);
PyThreadState_Delete(tcur);

// release the GIL as PyEval_InitThreads 
// implicitly acquires the GIL
PyEval_ReleaseLock();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate);

Which seems to works fine. But I think that this behavior of PyGILState_Ensure should be either documented or fixed.

Thanks,
Kuno

msg57619 - (view) Author: Greg Chapman (glchapman21) Date: 2007-11-18 17:58
In my embedding, I use the following (adapting the example above):

// initialize the Python interpreter
Py_Initialize();

PyEval_InitThreads();

/* Swap out and return current thread state and release the GIL */
PyThreadState tstate = PyEval_SaveThread();

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

PyRun_SimpleString("import random\n");

PyGILState_Release(gstate); 


You don't have to free the tstate returned by PyEval_SaveThread because
it is the thread state of the main thread (as established during
Py_Initialize), and so it will be freed during Python's shut-down.

I think in general you should rarely need to call PyEval_ReleaseLock
directly; instead use PyEval_SaveThread, the Py_BEGIN_ALLOW_THREADS
macro, or PyGILState_Release (as appropriate).  The documentation should
probably say as much.
msg109823 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-07-10 06:31
Please check whether this is still an issue in 3.1, so that there is still an issue for 3.2.
msg109832 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-07-10 07:36
This is still the case: the documentation should mention that PyEval_ReleaseLock() is not the correct function to release "the GIL", both the interpreter lock *and* the current thread state have to be released.
msg109865 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-07-10 12:36
I'm not even sure what PyEval_AcquireLock() and PyEval_ReleaseLock() are good for. Perhaps they should be deprecated.
msg129695 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-02-28 13:10
Given the deprecation of PyEval_ReleaseLock in issue10913, should this be closed as "out of date"?
msg129710 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-02-28 14:48
> Given the deprecation of PyEval_ReleaseLock in issue10913, should this 
> be closed as "out of date"?

Indeed :)
History
Date User Action Args
2011-02-28 14:48:10pitrousetstatus: open -> closed
nosy: georg.brandl, terry.reedy, amaury.forgeotdarc, ncoghlan, pitrou, kunoospald, urBan_dK, glchapman21, docs@python
messages: + msg129710

resolution: out of date
stage: test needed ->
2011-02-28 13:10:30ncoghlansetnosy: + ncoghlan
messages: + msg129695
2010-07-10 12:36:04pitrousetnosy: + pitrou
messages: + msg109865
2010-07-10 07:36:34amaury.forgeotdarcsetnosy: + amaury.forgeotdarc, docs@python
messages: + msg109832

assignee: docs@python
components: - Interpreter Core
2010-07-10 07:32:02amaury.forgeotdarclinkissue1147646 superseder
2010-07-10 06:31:37terry.reedysetnosy: + terry.reedy

messages: + msg109823
versions: + Python 3.2, - Python 2.6, Python 3.0
2009-04-12 12:23:58georg.brandlsetassignee: georg.brandl -> (no value)
2009-04-07 04:05:19ajaksu2setnosy: + georg.brandl
versions: + Python 2.6, Python 3.0, - Python 2.5
assignee: georg.brandl
components: + Documentation
type: enhancement
stage: test needed
2007-11-18 17:58:04glchapman21setnosy: + glchapman21
messages: + msg57619
2007-11-08 23:52:54urBan_dKsetnosy: + urBan_dK
2007-05-16 19:46:29kunoospaldcreate