Message244211
Using pid namespacing it is possible to have multiple processes with the same pid. "semlock_new" creates a semaphore file with the template "/dev/shm/mp{pid}-{counter}". This can conflict if the same semaphore file already exists due to another Python process have the same pid.
This bug has been fixed in Python3: https://bugs.python.org/issue8713. However, that patch is very large (40 files, ~4.4k changed lines) and only incidentally fixes this bug while introducing a large backwards-incompatible refactoring and feature addition.
The following small patch to just _multiprocessing/semaphore.c fixes the problem by using the system clock and retrying to avoid conflicts:
--- a/Modules/_multiprocessing/semaphore.c
+++ b/Modules/_multiprocessing/semaphore.c
@@ -7,6 +7,7 @@
*/
#include "multiprocessing.h"
+#include <time.h>
enum { RECURSIVE_MUTEX, SEMAPHORE };
@@ -419,7 +420,7 @@ semlock_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
char buffer[256];
SEM_HANDLE handle = SEM_FAILED;
- int kind, maxvalue, value;
+ int kind, maxvalue, value, try;
PyObject *result;
static char *kwlist[] = {"kind", "value", "maxvalue", NULL};
static int counter = 0;
@@ -433,10 +434,24 @@ semlock_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
return NULL;
}
- PyOS_snprintf(buffer, sizeof(buffer), "/mp%ld-%d", (long)getpid(), counter++);
+ /* With pid namespaces, we may have multiple processes with the same pid.
+ * Instead of relying on the pid to be unique, we use the microseconds time
+ * to attempt to a unique filename. */
+ for (try = 0; try < 100; ++try) {
+ struct timespec tv;
+ long arbitrary = clock_gettime(CLOCK_REALTIME, &tv) ? 0 : tv.tv_nsec;
+ PyOS_snprintf(buffer, sizeof(buffer), "/mp%ld-%d-%ld",
+ (long)getpid(),
+ counter++,
+ arbitrary);
+ SEM_CLEAR_ERROR();
+ handle = SEM_CREATE(buffer, value, maxvalue);
+ if (handle != SEM_FAILED)
+ break;
+ else if (errno != EEXIST)
+ goto failure;
+ }
- SEM_CLEAR_ERROR();
- handle = SEM_CREATE(buffer, value, maxvalue);
/* On Windows we should fail if GetLastError()==ERROR_ALREADY_EXISTS */
if (handle == SEM_FAILED || SEM_GET_LAST_ERROR() != 0)
goto failure; |
|
Date |
User |
Action |
Args |
2015-05-27 18:22:59 | Paul Hobbs | set | recipients:
+ Paul Hobbs |
2015-05-27 18:22:59 | Paul Hobbs | set | messageid: <1432750979.18.0.523128349285.issue24303@psf.upfronthosting.co.za> |
2015-05-27 18:22:59 | Paul Hobbs | link | issue24303 messages |
2015-05-27 18:22:59 | Paul Hobbs | create | |
|