classification
Title: Multiprocessing package build problem on Solaris 10
Type: compile error Stage:
Components: Extension Modules Versions: Python 3.0, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: jnoller, loewis, osvenskan, pitrou, roudkerk, skip.montanaro
Priority: high Keywords: patch

Created on 2008-06-14 19:36 by jnoller, last changed 2009-04-02 02:45 by jnoller. This issue is now closed.

Files
File name Uploaded Description Edit
issue_3110.patch jnoller, 2008-09-03 17:48
explore_sem_value_max.c osvenskan, 2008-12-18 01:50
Messages (13)
msg68210 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-06-14 19:36
Per skip's email:
FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does 
define
_SEM_VALUE_MAX in sys/params.h.

   .../Modules/_multiprocessing/multiprocessing.c: In function 
'init_multiprocessing':
   .../Modules/_multiprocessing/multiprocessing.c:253: error: 
'SEM_VALUE_MAX' undeclared (first use in this function)
   .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each 
undeclared identifier is reported only once
   .../Modules/_multiprocessing/multiprocessing.c:253: error: for each 
function it appears in.)

On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX.  I 
used
a little cpp action to define it:

   #ifndef SEM_VALUE_MAX
   #  ifdef _SEM_VALUE_MAX
   #    define SEM_VALUE_MAX _SEM_VALUE_MAX
   #  else
   #    define SEM_VALUE_MAX INT_MAX
   #  endif
   #endif
msg68330 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-06-17 15:59
Jesse> Per skip's email:
    Jesse> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does 
    Jesse> define
    Jesse> _SEM_VALUE_MAX in sys/params.h.

    ...

Thanks for submitting the bug report.  I wonder why the processing module
installs on Solaris10 but multiprocessing fails to compile.  Did the
SEM_VALUE_MAX code change between the two variants?

Skip
msg71985 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-08-26 16:11
FWIW, this is what I find on a Solaris box.

./sys/param.h:#define   _SEM_VALUE_MAX          INT_MAX
./sys/sysconfig.h:#define       _CONFIG_SEM_VALUE_MAX   21      /* max.
value a semaphore may have */
./sys/unistd.h:#define  _SC_SEM_VALUE_MAX               37
./limits.h:#define      _POSIX_SEM_VALUE_MAX    32767
msg72395 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-03 16:54
Raising this to RB, we should not RC without the MP module properly 
compiling
msg72399 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-09-03 17:06
I suggest committing a patch which falls back to _SEM_VALUE_MAX and see
how the Solaris buildbot reacts.
msg72403 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-03 17:48
Anyone mind reviewing the attached patch? This should resolve the solaris 
compile issue. I used skip's suggested code - I removed the #ifdef solaris 
at AP's suggestion.
msg72405 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2008-09-03 18:04
I recompiled and tested multiprocessing both under Windows and Linux
with this patch, no problems detected. +1 for applying it.
msg72408 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2008-09-03 18:22
I can confirm that Jesse's patch allows multiprocessing to compile on
Solaris 10.  Note, however, that there are other symbolic constants
defined which contain "SEM_VALUE_MAX", all with widely differing
underlying values:

    % find /usr/include -name '*.h' | xargs egrep SEM_VALUE_MAX
    /usr/include/sys/param.h:#define        _SEM_VALUE_MAX          INT_MAX
    /usr/include/sys/sysconfig.h:#define    _CONFIG_SEM_VALUE_MAX   21 
    /* max. value a semaphore may have */
    /usr/include/sys/unistd.h:#define       _SC_SEM_VALUE_MAX          
    37
    /usr/include/limits.h:#define   _POSIX_SEM_VALUE_MAX    32767

How do we know that _SEM_VALUE_MAX is the proper rvalue to use when
#define-ing SEM_VALUE_MAX?

Richard, as the author of the original processing module do you have
something to contribute to this discussion?  You've been completely
silent on this issue.
msg72409 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-03 18:23
Skip - Richard has been unavailable a good chunk of the summer. I don't 
know when he will be online again.
msg72410 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008-09-03 18:24
I've committed the patch in r66184 on trunk, 66185 py3k. Skip raises a 
good point, therefore I'll leave this open but lower from a blocker.
msg73011 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-09-11 09:51
According to POSIX, if no determinate value for SEM_VALUE_MAX can be
given, the actual value should be queried with
sysconf(_SC_SEM_VALUE_MAX), and SEM_VALUE_MAX should not be defined;
this is in particular the case when the value depends on the system
configuration (such as available memory). _POSIX_SEM_VALUE_MAX specifies
the minimum value guaranteed by POSIX.

Now, it is not plausible why SEM_VALUE_MAX should vary by installation,
as it just depends on what integer type is used to represent the
counter. So more likely, Sun wrote its header files at a time when the
spec was not finished, so they didn't want to clutter the global namespace.

IOW, I think the patch is fine as it stands. If you are over-cautious,
you should test whether _SC_SEM_VALUE_MAX is defined, and use sysconf in
that case, and only fall back to _SEM_VALUE_MAX, then
_POSIX_SEM_VALUE_MAX, then fail, if sysconf isn't available.

If you do make this change, please stop using Py_BuildValue to convert a
C int to a Python int; use PyInt_FromLong instead.
msg78006 - (view) Author: Philip Semanchuk (osvenskan) * Date: 2008-12-18 01:50
I'm facing the same problem (getting a good definition of SEM_VALUE_MAX)
for my posix_ipc extension. The patch presented here will get the
compiler to shut up on OpenSolaris, but not for the reason you think. At
least, that's the way I see it.

On OpenSolaris (2008.05 is what I'm testing with), _SEM_VALUE_MAX is
indeed #defined in sys/param.h, but it's inside this #ifdef on line 322:

#if (defined(_KERNEL) || defined(_KMEMUSER))

Since multiprocessing.c doesn't #define either of these, both
SEM_VALUE_MAX and _SEM_VALUE_MAX will remain undefined and so the patch
will always fall back to INT_MAX.

IMHO, given the choice between #defining _KERNEL or _KMEMUSER and
calling sysconf(_SC_SEM_VALUE_MAX), I feel safer doing the latter. 

I did a survey of all the operating systems to which I have access
(OpenSolaris 2008.05, OS X 10.5.5, RHEL?, Ubuntu 8.04, and FreeBSD 7).
OpenSolaris is the only one that doesn't #define SEM_VALUE_MAX, and on
all systems except for the possibly Red Hattish one,
sysconf(_SC_SEM_VALUE_MAX) returned the same value as the definition of
SEM_VALUE_MAX. I attached my code & results.
msg85141 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009-04-02 02:45
Additional protection checked in in r71022
History
Date User Action Args
2009-04-02 02:45:22jnollersetstatus: open -> closed
resolution: fixed
messages: + msg85141
2008-12-18 01:50:47osvenskansetfiles: + explore_sem_value_max.c
messages: + msg78006
2008-12-17 19:09:40osvenskansetnosy: + osvenskan
2008-09-11 09:52:01loewissetkeywords: - needs review
nosy: + loewis
messages: + msg73011
2008-09-03 18:24:55jnollersetpriority: release blocker -> high
messages: + msg72410
2008-09-03 18:23:34jnollersetmessages: + msg72409
2008-09-03 18:22:08skip.montanarosetmessages: + msg72408
2008-09-03 18:04:45pitrousetmessages: + msg72405
2008-09-03 18:00:49jnollersetkeywords: + needs review
2008-09-03 17:48:06jnollersetfiles: + issue_3110.patch
keywords: + patch
messages: + msg72403
2008-09-03 17:06:52pitrousetmessages: + msg72399
2008-09-03 16:54:08jnollersetpriority: high -> release blocker
messages: + msg72395
2008-08-26 16:11:59pitrousetpriority: high
nosy: + pitrou
messages: + msg71985
2008-06-19 13:36:01jnollersetassignee: jnoller
2008-06-17 16:00:00skip.montanarosetmessages: + msg68330
2008-06-14 19:36:15jnollercreate