Hello,

there's some issues compiling the multiprocessing module on the SunOS I
have here, where CMSG_LEN, CMSG_ALIGN, CMSG_SPACE and sem_timedwait are
absent.

$ uname -av
SunOS xxxxxxx 5.9 Generic_117171-15 sun4u sparc SUNW,Sun-Fire-V440

it looks like simply defining the first three macros like this works:
#ifndef CMSG_SPACE
#   define CMSG_SPACE(len) (CMSG_ALIGN(sizeof(struct
cmsghdr))+CMSG_ALIGN(len))
#endif

#ifndef CMSG_ALIGN
#   ifdef __sun__
#       define CMSG_ALIGN _CMSG_DATA_ALIGN
#   else
#       define CMSG_ALIGN(len) (((len)+sizeof(long)-1) & ~(sizeof(long)-1))
#   endif
#endif

#ifndef CMSG_LEN
#   define CMSG_LEN(len) (CMSG_ALIGN(sizeof(struct cmsghdr))+(len))
#endif

(from http://mailman.videolan.org/pipermail/vlc-devel/2006-May/024400.html )

and setting HAVE_SEM_TIMEDWAIT=0 in setup.py does the trick for the last
one.  Presumably, adding something like 

    elif platform.startswith('sunos5'):
        macros = dict(
                HAVE_SEM_OPEN=1,
                HAVE_SEM_TIMEDWAIT=0,
                HAVE_FD_TRANSFER=1
                )
        libraries = ['rt']

in setup.py should work, but I have no other SunOS machines to test this on.

thanks

Is this still an issue with more recent versions of Python?

Yes, it is. I encountered it at Solaris9 with python 2.7.1.

One more thing, as I tried to rebuild whole python I've encountered on following problem:

building '_multiprocessing' extension
gcc -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -DHAVE_SEM_OPEN=1 -DHAVE_FD_TRANSFER=1 -DHAVE_SEM_TIMEDWAIT=0 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/local/include -I/home/malyska/bld/python_sol10/Python-2.7.1/Include -I/home/malyska/bld/python_sol10/Python-2.7.1 -c /home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/multiprocessing.c -o build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/multiprocessing.o
gcc -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -DHAVE_SEM_OPEN=1 -DHAVE_FD_TRANSFER=1 -DHAVE_SEM_TIMEDWAIT=0 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/local/include -I/home/malyska/bld/python_sol10/Python-2.7.1/Include -I/home/malyska/bld/python_sol10/Python-2.7.1 -c /home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/socket_connection.c -o build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/socket_connection.o
gcc -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -DHAVE_SEM_OPEN=1 -DHAVE_FD_TRANSFER=1 -DHAVE_SEM_TIMEDWAIT=0 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/local/include -I/home/malyska/bld/python_sol10/Python-2.7.1/Include -I/home/malyska/bld/python_sol10/Python-2.7.1 -c /home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/semaphore.c -o build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/semaphore.o
gcc -shared build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/multiprocessing.o build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/socket_connection.o build/temp.solaris-2.9-sun4u-2.7/home/malyska/bld/python_sol10/Python-2.7.1/Modules/_multiprocessing/semaphore.o -L/home/malyska/bld/python_sol10/install/lib -L/usr/local/lib -o build/lib.solaris-2.9-sun4u-2.7/_multiprocessing.so
*** WARNING: renaming "_multiprocessing" since importing it failed: ld.so.1: python: fatal: relocation error: file build/lib.solaris-2.9-sun4u-2.7/_multiprocessing.so: symbol sem_timedwait: referenced symbol not found

so I had to commented out HAVE_SEM_TIMEDWAIT from setup.py, see:
             elif platform.startswith('sunos5'):
                  macros = dict(
                      HAVE_SEM_OPEN=1,
                      HAVE_FD_TRANSFER=1
                      )
                      #HAVE_SEM_TIMEDWAIT=0,
                  libraries = ['rt']
thanks

Hello,

> there's some issues compiling the multiprocessing module on the SunOS
> I have here, where CMSG_LEN, CMSG_ALIGN, CMSG_SPACE and sem_timedwait
> are absent.

CMSG_LEN and friends should be defined by <sys/socket.h> (as required by POSIX). SunOS 5.10 man page lists them:

http://download.oracle.com/docs/cd/E19253-01/816-5173/socket.h-3head/index.html

But not the SunOS 5.9 version:
http://ewp.rpi.edu/hartford/webgen/sysdocs/C/solaris_9/SUNWaman/hman3head/socket.3head.html

> it looks like simply defining the first three macros like this works

It works, but it's probably not a good idea: if the headers don't define CMSG_LEN and friends, then FD passing will probably not work.
It'd be better to not compile multiprocessing_(sendfd|recvfd) if CMSG_LEN is not defined (patch attached).

> sem_timedwait are absent.

Hmmm.
Do you have the compilation's log?
Normally, if sem_timedwait isn't available, HAVE_SEM_TIMEDWAIT shouldn't be defined, and we should fallback to sem_trywait (by the way, calling sem_trywait multiple times until the timeout expires is not the same has calling sem_timedwait: this will fail in case of heavy contention).
So this should build correctly.

And this seems even stranger when I read Sebastian's message:
"""
so I had to commented out HAVE_SEM_TIMEDWAIT from setup.py, see:
             elif platform.startswith('sunos5'):
                  macros = dict(
                      HAVE_SEM_OPEN=1,
                      HAVE_FD_TRANSFER=1
                      )
                      #HAVE_SEM_TIMEDWAIT=0,
                  libraries = ['rt']
"""

Makes sense.
If we define HAVE_SEMTIMEDWAIT=0, then code guarded by
#ifdef HAVE_SEMTIMEDWAIT

will be compiled, and the linker won't be able to resolve sem_timedwait.
The preprocessor just checks that the symbol is defined, not that it has a non-zero value.
To sum up: could someone with a SunOS box test the attached patch, and post the compilation logs if it still fails?

I can confirm that the compile completes with a multiprocessing module where it hasn't before--at least in SunOS 5.9.

New changeset 8c658b625475 by Charles-François Natali in branch '2.7':
Issue #4028: Make multiprocessing build on SunOS.
http://hg.python.org/cpython/rev/8c658b625475

New changeset 49e82c885d6b by Charles-François Natali in branch '3.2':
Issue #4028: Make multiprocessing build on SunOS.
http://hg.python.org/cpython/rev/49e82c885d6b

New changeset 4a27b5ab2710 by Charles-François Natali in branch 'default':
Null merge - Issue #4028: Make multiprocessing build on SunOS.
http://hg.python.org/cpython/rev/4a27b5ab2710

Thanks Craig.