classification
Title: _multiprocessing module builds incorrectly when POSIX semaphores are disabled
Type: compile error Stage: resolved
Components: Extension Modules Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: sbt Nosy List: Arfrever, jnoller, koobs, python-dev, sbt
Priority: normal Keywords: easy, patch

Created on 2014-06-10 10:33 by Arfrever, last changed 2014-08-25 06:56 by koobs. This issue is now closed.

Files
File name Uploaded Description Edit
freebsd-python34.log koobs, 2014-08-25 06:55
Messages (4)
msg220162 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-06-10 10:33
When POSIX semaphores are disabled (e.g. by unmounting /dev/shm on a Linux system), then _multiprocessing module builds with undefined symbol _PyMp_sem_unlink:

$ ./configure ...
...
checking whether POSIX semaphores are enabled... no
...
$ make
...
building '_multiprocessing' extension
creating build/temp.linux-x86_64-3.5/tmp/cpython/Modules/_multiprocessing
x86_64-pc-linux-gnu-gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -march=core2 -O2 -fno-ident -pipe -ggdb3 -Wall -Wpointer-sign -IModules/_multiprocessing -I./Include -I. -IInclude -I/usr/local/include -I/tmp/cpython/Include -I/tmp/cpython -c /tmp/cpython/Modules/_multiprocessing/multiprocessing.c -o build/temp.linux-x86_64-3.5/tmp/cpython/Modules/_multiprocessing/multiprocessing.o
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -Wl,--gc-sections -Wl,--hash-style=gnu -Wl,--sort-common build/temp.linux-x86_64-3.5/tmp/cpython/Modules/_multiprocessing/multiprocessing.o -L. -L/usr/local/lib -lpython3.5m -o build/lib.linux-x86_64-3.5/_multiprocessing.cpython-35m.so
*** WARNING: renaming "_multiprocessing" since importing it failed: build/lib.linux-x86_64-3.5/_multiprocessing.cpython-35m.so: undefined symbol: _PyMp_sem_unlink
...
Following modules built successfully but were removed because they could not be imported:
_multiprocessing


This problem was introduced in Python 3.4. This problem is absent in older versions of Python.
Potential fix:

--- Modules/_multiprocessing/multiprocessing.c
+++ Modules/_multiprocessing/multiprocessing.c
@@ -129,5 +129,7 @@
     {"send", multiprocessing_send, METH_VARARGS, ""},
 #endif
+#ifndef POSIX_SEMAPHORES_NOT_ENABLED
     {"sem_unlink", _PyMp_sem_unlink, METH_VARARGS, ""},
+#endif
     {NULL}
 };
msg224199 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-28 22:16
New changeset f618f6739200 by Richard Oudkerk in branch '3.4':
Issue #21704: Fix build error for _multiprocessing when semaphores
http://hg.python.org/cpython/rev/f618f6739200

New changeset a9637b29954d by Richard Oudkerk in branch 'default':
Issue #21704: Merge.
http://hg.python.org/cpython/rev/a9637b29954d
msg224227 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-29 14:36
New changeset 1a00be3d79bc by Ezio Melotti in branch '3.4':
#21704: remove duplicate name in Misc/ACKS.
http://hg.python.org/cpython/rev/1a00be3d79bc

New changeset 723e0a7c4914 by Ezio Melotti in branch 'default':
#21704: merge with 3.4.
http://hg.python.org/cpython/rev/723e0a7c4914
msg225868 - (view) Author: Kubilay Kocak (koobs) Date: 2014-08-25 06:55
For reference and breadcrumbs, this also occurs when:

checking for sem_open... yes
checking for sem_timedwait... yes
checking for sem_getvalue... yes
checking for sem_unlink... yes
<snip>
checking whether POSIX semaphores are enabled... no
checking for broken sem_getvalue... yes
<snip>
*** WARNING: renaming "_multiprocessing" since importing it failed: build/lib.freebsd-8.4-RELEASE-i386-3.4/_multiprocessing.so: Undefined symbol "_PyMp_sem_unlink"

This can occur on FreeBSD jails, among other things, when host & jail versions differ.

Attaching full QA log from RedPorts on FreeBSD 8.4-RELEASE
History
Date User Action Args
2014-08-25 06:56:07koobssetfiles: + freebsd-python34.log

messages: + msg225868
2014-07-29 14:38:12Arfreversetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-07-29 14:36:38python-devsetmessages: + msg224227
2014-07-28 22:16:33python-devsetnosy: + python-dev
messages: + msg224199
2014-07-08 16:27:05koobssetnosy: + koobs
2014-06-10 10:35:39Arfreversetkeywords: + patch
2014-06-10 10:33:24Arfrevercreate