This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "with" statement isn't thread-safe
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.1
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Abyx, amaury.forgeotdarc
Priority: normal Keywords:

Created on 2010-11-12 14:03 by Abyx, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (3)
msg121035 - (view) Author: Abyx (Abyx) Date: 2010-11-12 14:03
Code to reproduce the bug:

#include <python.h>
#include <windows.h>

DWORD WINAPI thread_fn(void* code)
{
	PyGILState_STATE state = PyGILState_Ensure();
	PyRun_SimpleString("with sync: print('.')\n");
	PyGILState_Release(state);
	return 0;
}

int main()
{
	PyEval_InitThreads();
	Py_Initialize();
	PyRun_SimpleString("import _thread\n");
	PyRun_SimpleString("sync = _thread.allocate_lock()\n");
	PyThreadState* mainstate = PyEval_SaveThread();

	HANDLE hThread1 = CreateThread(0, 0, thread_fn, 0, 0, 0);
	HANDLE hThread2 = CreateThread(0, 0, thread_fn, 0, 0, 0);
	WaitForSingleObject(hThread1, INFINITE);
	WaitForSingleObject(hThread2, INFINITE);

	PyEval_RestoreThread(mainstate);
	Py_Finalize();
}

------------ Output ------
.
.
Traceback (most recent call last):
  File "<string>", line 2, in <module>
NameError: name '_[1]' is not defined
----------- End of output -----

Probably both threads uses the same "_" global variable. First thread releases GIL in "print" function, then second thread overwrites "_", and then first thread raises the NameError accessing "_" later. Sometimes (not ever) the "sync" kept locked, and program deadlocks on next "sync.acquire()" call from another thread (if any).
msg121037 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2010-11-12 14:39
Agreed. There was a similar issue in python2.6 with list comprehensions.
The issue is that both threads run the code with the same globals dictionary, and top-level code use the same dict for locals and globals.

This is already fixed in 2.7 and 3.2, temporary variables are no more used.

Meanwhile, I can see two workarounds:
- Run the code in a python function; the _[1] variable will be local to the function, and distinct for each thread.
- Use PyRun_String(), and pass a different dictionary for each thread::

    PyObject *d, *v;
    d = PyDict_New();
    v = PyRun_String(command, Py_file_input, d, d);
    Py_XDECREF(v);
    Py_XDECREF(d);

But the 3.2 fix cannot be ported to 3.1: it needs a new opcode.
msg121041 - (view) Author: Abyx (Abyx) Date: 2010-11-12 16:02
Thanks.
History
Date User Action Args
2022-04-11 14:57:08adminsetgithub: 54602
2010-11-12 16:02:47Abyxsetstatus: pending -> closed

messages: + msg121041
2010-11-12 14:39:16amaury.forgeotdarcsetstatus: open -> pending

nosy: + amaury.forgeotdarc
messages: + msg121037

resolution: out of date
2010-11-12 14:03:55Abyxcreate