Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc. #76702

Closed
stratakis mannequin opened this issue Jan 8, 2018 · 23 comments
Closed

NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc. #76702

stratakis mannequin opened this issue Jan 8, 2018 · 23 comments
Labels
3.7 (EOL) end of life build The build process and cross-build extension-modules C modules in the Modules dir

Comments

@stratakis
Copy link
Mannequin

stratakis mannequin commented Jan 8, 2018

BPO 32521
Nosy @vstinner, @tiran, @benjaminp, @mcepl, @stratakis
PRs
  • bpo-32521: nis libtirpc #5137
  • [3.6] bpo-32521: nis libtirpc (GH-5137) #5165
  • [2.7] bpo-32521: nis libtirpc (GH-5137) #5166
  • bpo-32521: nis libnsl #5190
  • [3.6] bpo-32521: nis libnsl (GH-5190)  #5352
  • [2.7] bpo-32521: nis libnsl (GH-5190) #5353
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-03-16.23:59:31.765>
created_at = <Date 2018-01-08.16:35:03.650>
labels = ['extension-modules', 'build', '3.7']
title = 'NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc.'
updated_at = <Date 2020-03-16.23:59:31.764>
user = 'https://github.com/stratakis'

    bugs.python.org fields:

    activity = <Date 2020-03-16.23:59:31.764>
actor = 'cstratak'
assignee = 'none'
closed = True
closed_date = <Date 2020-03-16.23:59:31.765>
closer = 'cstratak'
components = ['Extension Modules']
creation = <Date 2018-01-08.16:35:03.650>
creator = 'cstratak'
dependencies = []
files = []
hgrepos = []
issue_num = 32521
keywords = ['patch']
message_count = 23.0
messages = ['309674', '309675', '309676', '309677', '309678', '309681', '309684', '309856', '309860', '309861', '309863', '309967', '310813', '310847', '310848', '311729', '321073', '321075', '321098', '321121', '321186', '321309', '364376']
nosy_count = 5.0
nosy_names = ['vstinner', 'christian.heimes', 'benjamin.peterson', 'mcepl', 'cstratak']
pr_nums = ['5137', '5165', '5166', '5190', '5352', '5353']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue32521'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']
    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jan 8, 2018

    Currently on the development branch of Fedora (28), an upstream change of glibc is being pushed where the Sun RPC support is removed from our downstream glibc package in favor of the libtirpc library. More details [0].

    As a result, when python is trying to build the nis extension, it will complain with:
fatal error: rpc/rpc.h: No such file or directory
#include <rpc/rpc.h>
          ^~~~~~~~~~~
compilation terminated.
 
[0] https://fedoraproject.org/wiki/Changes/SunRPCRemoval

    @stratakis stratakis mannequin added 3.8 only security fixes 3.7 (EOL) end of life extension-modules C modules in the Modules dir labels Jan 8, 2018
    @stratakis stratakis mannequin changed the title NIS module fails to build due to the remove of interfaces related to Sun RPC from glibc. NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc. Jan 8, 2018
    @vstinner
    Copy link
    Member

    vstinner commented Jan 8, 2018

    Do you know if installing libtirpc-devel does install required rpc/rpc.h / fixes the issue?

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jan 8, 2018

    Already tried. Unfortunately it doesn't.

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jan 8, 2018

    The header is located at /usr/include/tirpc/rpc/rpc.h

    @vstinner
    Copy link
    Member

    vstinner commented Jan 8, 2018

    The header is located at /usr/include/tirpc/rpc/rpc.h

    configure.ac should be modified to detect tirpc is installed. If yes, setup.py should add /usr/include/tirpc/ to includes to build the "nis" module.

    @tiran tiran self-assigned this Jan 8, 2018
    @tiran
    Copy link
    Member

    tiran commented Jan 8, 2018

    I modified setup.py to look for libtirpc. In case the library is detected, it's added as dependency and /usr/include/tirpc is included in search path.

    @tiran tiran removed the 3.8 only security fixes label Jan 8, 2018
    @tiran tiran removed their assignment Jan 8, 2018
    @benjaminp
    Copy link
    Contributor

    I think this is a duplicate of bpo-32007.

    @tiran
    Copy link
    Member

    tiran commented Jan 12, 2018

    New changeset f3031b8 by Christian Heimes in branch 'master':
    bpo-32521: nis libtirpc (bpo-5137)
    f3031b8

    @tiran
    Copy link
    Member

    tiran commented Jan 12, 2018

    New changeset d55d682 by Christian Heimes (Miss Islington (bot)) in branch '3.6':
    [3.6] bpo-32521: nis libtirpc (GH-5137) (bpo-5165)
    d55d682

    @tiran
    Copy link
    Member

    tiran commented Jan 12, 2018

    New changeset ab95b30 by Christian Heimes (Miss Islington (bot)) in branch '2.7':
    bpo-32521: nis libtirpc (GH-5137) (bpo-5166)
    ab95b30

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jan 12, 2018

    Unfortunately the yp_prot.h and ypclnt.h [0] headers have also moved to a different package.

    Currently they reside at /usr/include/nsl/rpcsvc/*.h

    [0]

    cpython/Modules/nismodule.c

    Line 18 in f3031b8

    #include <rpcsvc/yp_prot.h>

    [1] https://fedoraproject.org/wiki/Changes/NISIPv6

    @tiran
    Copy link
    Member

    tiran commented Jan 15, 2018

    It's not just the headers. The shared library is now /usr/lib64/nsl/libnsl.so.2 instead of /usr/lib64/libnsl.so.1.

    @tiran
    Copy link
    Member

    tiran commented Jan 26, 2018

    New changeset 29a7df7 by Christian Heimes in branch 'master':
    bpo-32521: nis libnsl (bpo-5190)
    29a7df7

    @tiran
    Copy link
    Member

    tiran commented Jan 27, 2018

    New changeset 12ae407 by Christian Heimes in branch '3.6':
    [3.6] bpo-32521: nis libnsl (GH-5190) (bpo-5352)
    12ae407

    @tiran
    Copy link
    Member

    tiran commented Jan 27, 2018

    New changeset 38487a0 by Christian Heimes in branch '2.7':
    [2.7] bpo-32521: nis libnsl (GH-5190) (bpo-5353)
    38487a0

    @tiran tiran closed this as completed Jan 27, 2018
    @tiran tiran added the build The build process and cross-build label Jan 27, 2018
    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Feb 6, 2018

    Reopenning the issue.

    python2 is still failing with the fix applied:

    gcc -pthread -fPIC -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv  -I. -IInclude -I/builddir/build/BUILD/Python-2.7.14/Include   -c /builddir/build/BUILD/Python-2.7.14/Modules/nismodule.c -o Modules/nismodule.o
/builddir/build/BUILD/Python-2.7.14/Modules/nismodule.c:17:10: fatal error: rpc/rpc.h: No such file or directory
 #include <rpc/rpc.h>
          ^~~~~~~~~~~
compilation terminated.
{standard input}: Assembler messages:
{standard input}: Error: .size expression for nismodule.c does not evaluate to a constant
make: *** [Makefile:1568: Modules/nismodule.o] Error 1
make: *** Waiting for unfinished jobs....

    @stratakis stratakis mannequin reopened this Feb 6, 2018
    @mcepl
    Copy link
    Mannequin

    mcepl mannequin commented Jul 5, 2018

    So, if this is a duplicate of bpo-32007, shouldn't it be closed?

    @mcepl
    Copy link
    Mannequin

    mcepl mannequin commented Jul 5, 2018

    It seems to be even more weird. Even with installed libnsl-devel (libnsl-devel-1.2.0-2.5.x86_64 on openSUSE/Tumbleweed) I get this in config.log:

    checking for t_open in -lnsl... no

    and truly, when grepping for t_open in /usr/include/rpc* I get nothing. When searching for t_open, it seems to be something from Solaris world.

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jul 5, 2018

    Matej is this about Python 2? Because the solution didn't actually work for Python 2 and on Fedora we had to implement a workaround [0]. Unfortunately there weren't enough free cycles so far to investigate further.

    [0] https://src.fedoraproject.org/rpms/python2/c/3056bfd92a4269ad8f9b57cab05af3125e87ca8c?branch=master

    @mcepl
    Copy link
    Mannequin

    mcepl mannequin commented Jul 5, 2018

    Matej is this about Python 2?

    I am currently ONLY on building python 3.7.0, nothing else bothers me at the moment.

    Let me summarize my findings, or what I think is the situation (of course, I could be completely wrong):

    I think the problem is that the Python build system expects libnsl to be the one which is found in Solaris and so it looks for its API. Except, the Linux one is different and doesn't provide the same API. And from there, it goes all down to hell.

    With Leap 42.3 (which has glibc-2.27-4.1, but no libnsl, so I guess NIS API is still inside of glibc; is it possible?), nis builds correctly and so it is included.

    OK, so I have no clue and it is all complete mess.

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Jul 6, 2018

    On Fedora after the fix we had also to BuildRequire libnsl2-devel and libtirpc-devel after the glibc change was implemented for the correct headers to be included within the buildroot.

    I am not familiar with suse's build system webui to find the logs, but are those packages provided by the distribution?

    @mcepl
    Copy link
    Mannequin

    mcepl mannequin commented Jul 9, 2018

    On Fedora after the fix we had also to BuildRequire libnsl2-devel and libtirpc-devel after the glibc change was implemented for the correct headers to be included within the buildroot.

    libnsl-devel requires libtirpc-devel here, so that isn't the issue.

    @stratakis
    Copy link
    Mannequin Author

    stratakis mannequin commented Mar 16, 2020

    Closing the issue as python2 is not receiving any more fixes and our downstream workaround is enough for it. Python3 is fine as well.

    @stratakis stratakis mannequin closed this as completed Mar 16, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life build The build process and cross-build extension-modules C modules in the Modules dir
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @benjaminp @tiran