Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing package build problem on Solaris 10 #47360

Closed
jnoller mannequin opened this issue Jun 14, 2008 · 13 comments
Closed

Multiprocessing package build problem on Solaris 10 #47360

jnoller mannequin opened this issue Jun 14, 2008 · 13 comments
Labels
build The build process and cross-build extension-modules C modules in the Modules dir

Comments

@jnoller
Copy link
Mannequin

jnoller mannequin commented Jun 14, 2008

BPO 3110
Nosy @loewis, @smontanaro, @pitrou, @osvenskan
Files
  • issue_3110.patch
  • explore_sem_value_max.c
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2009-04-02.02:45:22.428>
    created_at = <Date 2008-06-14.19:36:15.855>
    labels = ['extension-modules', 'build']
    title = 'Multiprocessing package build problem on Solaris 10'
    updated_at = <Date 2009-04-02.02:45:22.426>
    user = 'https://bugs.python.org/jnoller'

    bugs.python.org fields:

    activity = <Date 2009-04-02.02:45:22.426>
    actor = 'jnoller'
    assignee = 'jnoller'
    closed = True
    closed_date = <Date 2009-04-02.02:45:22.428>
    closer = 'jnoller'
    components = ['Extension Modules']
    creation = <Date 2008-06-14.19:36:15.855>
    creator = 'jnoller'
    dependencies = []
    files = ['11361', '12391']
    hgrepos = []
    issue_num = 3110
    keywords = ['patch']
    message_count = 13.0
    messages = ['68210', '68330', '71985', '72395', '72399', '72403', '72405', '72408', '72409', '72410', '73011', '78006', '85141']
    nosy_count = 6.0
    nosy_names = ['loewis', 'skip.montanaro', 'pitrou', 'osvenskan', 'roudkerk', 'jnoller']
    pr_nums = []
    priority = 'high'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = 'compile error'
    url = 'https://bugs.python.org/issue3110'
    versions = ['Python 2.6', 'Python 3.0']

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Jun 14, 2008

    Per skip's email:
    FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does
    define
    _SEM_VALUE_MAX in sys/params.h.

    .../Modules/_multiprocessing/multiprocessing.c: In function
    'init_multiprocessing':
    .../Modules/_multiprocessing/multiprocessing.c:253: error:
    'SEM_VALUE_MAX' undeclared (first use in this function)
    .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each
    undeclared identifier is reported only once
    .../Modules/_multiprocessing/multiprocessing.c:253: error: for each
    function it appears in.)

    On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I
    used
    a little cpp action to define it:

       #ifndef SEM_VALUE_MAX
       #  ifdef _SEM_VALUE_MAX
       #    define SEM_VALUE_MAX _SEM_VALUE_MAX
       #  else
       #    define SEM_VALUE_MAX INT_MAX
       #  endif
       #endif

    @jnoller jnoller mannequin added extension-modules C modules in the Modules dir build The build process and cross-build labels Jun 14, 2008
    @smontanaro
    Copy link
    Contributor

    Jesse> Per skip's email:
    Jesse> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does
    Jesse> define
    Jesse> _SEM_VALUE_MAX in sys/params.h.

    ...
    

    Thanks for submitting the bug report. I wonder why the processing module
    installs on Solaris10 but multiprocessing fails to compile. Did the
    SEM_VALUE_MAX code change between the two variants?

    Skip

    @jnoller jnoller mannequin self-assigned this Jun 19, 2008
    @pitrou
    Copy link
    Member

    pitrou commented Aug 26, 2008

    FWIW, this is what I find on a Solaris box.

    ./sys/param.h:#define _SEM_VALUE_MAX INT_MAX
    ./sys/sysconfig.h:#define _CONFIG_SEM_VALUE_MAX 21 /* max.
    value a semaphore may have */
    ./sys/unistd.h:#define _SC_SEM_VALUE_MAX 37
    ./limits.h:#define _POSIX_SEM_VALUE_MAX 32767

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Sep 3, 2008

    Raising this to RB, we should not RC without the MP module properly
    compiling

    @jnoller jnoller mannequin added the release-blocker label Sep 3, 2008
    @pitrou
    Copy link
    Member

    pitrou commented Sep 3, 2008

    I suggest committing a patch which falls back to _SEM_VALUE_MAX and see
    how the Solaris buildbot reacts.

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Sep 3, 2008

    Anyone mind reviewing the attached patch? This should resolve the solaris
    compile issue. I used skip's suggested code - I removed the #ifdef solaris
    at AP's suggestion.

    @pitrou
    Copy link
    Member

    pitrou commented Sep 3, 2008

    I recompiled and tested multiprocessing both under Windows and Linux
    with this patch, no problems detected. +1 for applying it.

    @smontanaro
    Copy link
    Contributor

    I can confirm that Jesse's patch allows multiprocessing to compile on
    Solaris 10. Note, however, that there are other symbolic constants
    defined which contain "SEM_VALUE_MAX", all with widely differing
    underlying values:

    % find /usr/include -name '*.h' | xargs egrep SEM_VALUE_MAX
    /usr/include/sys/param.h:#define        _SEM_VALUE_MAX          INT_MAX
    /usr/include/sys/sysconfig.h:#define    _CONFIG_SEM_VALUE_MAX   21 
    /* max. value a semaphore may have */
    /usr/include/sys/unistd.h:#define       _SC_SEM_VALUE_MAX          
    37
    /usr/include/limits.h:#define   _POSIX_SEM_VALUE_MAX    32767
    

    How do we know that _SEM_VALUE_MAX is the proper rvalue to use when
    #define-ing SEM_VALUE_MAX?

    Richard, as the author of the original processing module do you have
    something to contribute to this discussion? You've been completely
    silent on this issue.

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Sep 3, 2008

    Skip - Richard has been unavailable a good chunk of the summer. I don't
    know when he will be online again.

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Sep 3, 2008

    I've committed the patch in r66184 on trunk, 66185 py3k. Skip raises a
    good point, therefore I'll leave this open but lower from a blocker.

    @jnoller jnoller mannequin removed the release-blocker label Sep 3, 2008
    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Sep 11, 2008

    According to POSIX, if no determinate value for SEM_VALUE_MAX can be
    given, the actual value should be queried with
    sysconf(_SC_SEM_VALUE_MAX), and SEM_VALUE_MAX should not be defined;
    this is in particular the case when the value depends on the system
    configuration (such as available memory). _POSIX_SEM_VALUE_MAX specifies
    the minimum value guaranteed by POSIX.

    Now, it is not plausible why SEM_VALUE_MAX should vary by installation,
    as it just depends on what integer type is used to represent the
    counter. So more likely, Sun wrote its header files at a time when the
    spec was not finished, so they didn't want to clutter the global namespace.

    IOW, I think the patch is fine as it stands. If you are over-cautious,
    you should test whether _SC_SEM_VALUE_MAX is defined, and use sysconf in
    that case, and only fall back to _SEM_VALUE_MAX, then
    _POSIX_SEM_VALUE_MAX, then fail, if sysconf isn't available.

    If you do make this change, please stop using Py_BuildValue to convert a
    C int to a Python int; use PyInt_FromLong instead.

    @osvenskan
    Copy link
    Mannequin

    osvenskan mannequin commented Dec 18, 2008

    I'm facing the same problem (getting a good definition of SEM_VALUE_MAX)
    for my posix_ipc extension. The patch presented here will get the
    compiler to shut up on OpenSolaris, but not for the reason you think. At
    least, that's the way I see it.

    On OpenSolaris (2008.05 is what I'm testing with), _SEM_VALUE_MAX is
    indeed #defined in sys/param.h, but it's inside this #ifdef on line 322:

    #if (defined(_KERNEL) || defined(_KMEMUSER))
    

    Since multiprocessing.c doesn't #define either of these, both
    SEM_VALUE_MAX and _SEM_VALUE_MAX will remain undefined and so the patch
    will always fall back to INT_MAX.

    IMHO, given the choice between #defining _KERNEL or _KMEMUSER and
    calling sysconf(_SC_SEM_VALUE_MAX), I feel safer doing the latter.

    I did a survey of all the operating systems to which I have access
    (OpenSolaris 2008.05, OS X 10.5.5, RHEL?, Ubuntu 8.04, and FreeBSD 7).
    OpenSolaris is the only one that doesn't #define SEM_VALUE_MAX, and on
    all systems except for the possibly Red Hattish one,
    sysconf(_SC_SEM_VALUE_MAX) returned the same value as the definition of
    SEM_VALUE_MAX. I attached my code & results.

    @jnoller
    Copy link
    Mannequin Author

    jnoller mannequin commented Apr 2, 2009

    Additional protection checked in in r71022

    @jnoller jnoller mannequin closed this as completed Apr 2, 2009
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    build The build process and cross-build extension-modules C modules in the Modules dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants