classification
Title: NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc.
Type: compile error Stage: resolved
Components: Extension Modules Versions: Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, christian.heimes, cstratak, mcepl, vstinner
Priority: normal Keywords: patch

Created on 2018-01-08 16:35 by cstratak, last changed 2018-07-09 11:28 by mcepl.

Pull Requests
URL Status Linked Edit
PR 5137 merged christian.heimes, 2018-01-08 17:03
PR 5165 merged python-dev, 2018-01-12 14:26
PR 5166 merged python-dev, 2018-01-12 14:27
PR 5190 merged christian.heimes, 2018-01-15 11:54
PR 5352 merged christian.heimes, 2018-01-26 22:32
PR 5353 merged christian.heimes, 2018-01-26 22:36
Messages (22)
msg309674 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-01-08 16:35
Currently on the development branch of Fedora (28), an upstream change of glibc is being pushed where the Sun RPC support is removed from our downstream glibc package in favor of the libtirpc library. More details [0].

As a result, when python is trying to build the nis extension, it will complain with:
fatal error: rpc/rpc.h: No such file or directory
#include <rpc/rpc.h>
          ^~~~~~~~~~~
compilation terminated.
 
[0] https://fedoraproject.org/wiki/Changes/SunRPCRemoval
msg309675 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-08 16:41
Do you know if installing libtirpc-devel does install required rpc/rpc.h / fixes the issue?
msg309676 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-01-08 16:43
Already tried. Unfortunately it doesn't.
msg309677 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-01-08 16:45
The header is located at /usr/include/tirpc/rpc/rpc.h
msg309678 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-01-08 16:48
> The header is located at /usr/include/tirpc/rpc/rpc.h

configure.ac should be modified to detect tirpc is installed. If yes, setup.py should add /usr/include/tirpc/ to includes to build the "nis" module.
msg309681 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-08 19:45
I modified setup.py to look for libtirpc. In case the library is detected, it's added as dependency and /usr/include/tirpc is included in search path.
msg309684 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2018-01-09 05:54
I think this is a duplicate of #32007.
msg309856 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-12 14:26
New changeset f3031b8a7ad71d3b6ed05da7f3041d9efbe773cf by Christian Heimes in branch 'master':
bpo-32521: nis libtirpc (#5137)
https://github.com/python/cpython/commit/f3031b8a7ad71d3b6ed05da7f3041d9efbe773cf
msg309860 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-12 15:29
New changeset d55d6825d7bd0fc120c95d579065623ce6392a2f by Christian Heimes (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-32521: nis libtirpc (GH-5137) (#5165)
https://github.com/python/cpython/commit/d55d6825d7bd0fc120c95d579065623ce6392a2f
msg309861 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-12 15:35
New changeset ab95b3074ee43098edf3f23b07fb18ef57ee614d by Christian Heimes (Miss Islington (bot)) in branch '2.7':
bpo-32521: nis libtirpc (GH-5137) (#5166)
https://github.com/python/cpython/commit/ab95b3074ee43098edf3f23b07fb18ef57ee614d
msg309863 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-01-12 17:03
Unfortunately the yp_prot.h and ypclnt.h [0] headers have also moved to a different package.

Currently they reside at /usr/include/nsl/rpcsvc/*.h

[0] https://github.com/python/cpython/blob/f3031b8a7ad71d3b6ed05da7f3041d9efbe773cf/Modules/nismodule.c#L18
[1] https://fedoraproject.org/wiki/Changes/NISIPv6
msg309967 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-15 11:56
It's not just the headers. The shared library is now /usr/lib64/nsl/libnsl.so.2 instead of /usr/lib64/libnsl.so.1.
msg310813 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-26 22:28
New changeset 29a7df78277447cf6b898dfa0b1b42f8da7abc0c by Christian Heimes in branch 'master':
bpo-32521: nis libnsl (#5190)
https://github.com/python/cpython/commit/29a7df78277447cf6b898dfa0b1b42f8da7abc0c
msg310847 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-27 08:39
New changeset 12ae40766ea39f09c2ddf6842017ee80f7f3d00b by Christian Heimes in branch '3.6':
[3.6] bpo-32521: nis libnsl (GH-5190) (#5352)
https://github.com/python/cpython/commit/12ae40766ea39f09c2ddf6842017ee80f7f3d00b
msg310848 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-01-27 08:39
New changeset 38487a05cce32536b49dd6c89df7dc85b96eaef4 by Christian Heimes in branch '2.7':
[2.7] bpo-32521: nis libnsl (GH-5190) (#5353)
https://github.com/python/cpython/commit/38487a05cce32536b49dd6c89df7dc85b96eaef4
msg311729 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-02-06 15:55
Reopenning the issue.

python2 is still failing with the fix applied:

gcc -pthread -fPIC -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -mcet -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv  -I. -IInclude -I/builddir/build/BUILD/Python-2.7.14/Include   -c /builddir/build/BUILD/Python-2.7.14/Modules/nismodule.c -o Modules/nismodule.o
/builddir/build/BUILD/Python-2.7.14/Modules/nismodule.c:17:10: fatal error: rpc/rpc.h: No such file or directory
 #include <rpc/rpc.h>
          ^~~~~~~~~~~
compilation terminated.
{standard input}: Assembler messages:
{standard input}: Error: .size expression for nismodule.c does not evaluate to a constant
make: *** [Makefile:1568: Modules/nismodule.o] Error 1
make: *** Waiting for unfinished jobs....
msg321073 - (view) Author: Matej Cepl (mcepl) * Date: 2018-07-05 05:08
So, if this is a duplicate of #32007, shouldn't it be closed?
msg321075 - (view) Author: Matej Cepl (mcepl) * Date: 2018-07-05 05:14
It seems to be even more weird. Even with installed libnsl-devel (libnsl-devel-1.2.0-2.5.x86_64 on openSUSE/Tumbleweed) I get this in config.log:

checking for t_open in -lnsl... no

and truly, when grepping for t_open in /usr/include/rpc* I get nothing. When searching for t_open, it seems to be something from Solaris world.
msg321098 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-07-05 11:31
Matej is this about Python 2? Because the solution didn't actually work for Python 2 and on Fedora we had to implement a workaround [0]. Unfortunately there weren't enough free cycles so far to investigate further.

[0] https://src.fedoraproject.org/rpms/python2/c/3056bfd92a4269ad8f9b57cab05af3125e87ca8c?branch=master
msg321121 - (view) Author: Matej Cepl (mcepl) * Date: 2018-07-05 17:50
> Matej is this about Python 2?

I am currently ONLY on building python 3.7.0, nothing else bothers me at the moment.

Let me summarize my findings, or what I think is the situation (of course, I could be completely wrong):

 * See https://build.opensuse.org/package/show/devel:languages:python:Factory/python3 ... Tumbleweed and Leap 15  (which all have libnsl) fail to be build with libnsl-devel installed. It seems to me that it somehow tries to build nismodule.c, but fails in configure (see above), so it doesn't end up well.

I think the problem is that the Python build system expects libnsl to be the one which is found in Solaris and so it looks for its API. Except, the Linux one is different and doesn't provide the same API. And from there, it goes all down to hell.

With Leap 42.3 (which has glibc-2.27-4.1, but no libnsl, so I guess NIS API is still inside of glibc; is it possible?), nis builds correctly and so it is included.

* Concerning Python 2. I don't see any problems with it. https://build.opensuse.org/package/show/devel:languages:python:Factory/python shows https://is.gd/sbwIf6 that glibc is the same glibc-2.27-4.1, libnsl-devel-1.2.0-2.1, ./configure still fails, but http://download.opensuse.org/repositories/devel:/languages:/python:/Factory/openSUSE_Tumbleweed/x86_64/python-base-2.7.15-112.1.x86_64.rpm still contains /usr/lib64/python2.7/lib-dynload/nis.so module.

OK, so I have no clue and it is all complete mess.
msg321186 - (view) Author: Charalampos Stratakis (cstratak) * Date: 2018-07-06 17:38
On Fedora after the fix we had also to BuildRequire libnsl2-devel and libtirpc-devel after the glibc change was implemented for the correct headers to be included within the buildroot.

I am not familiar with suse's build system webui to find the logs, but are those packages provided by the distribution?
msg321309 - (view) Author: Matej Cepl (mcepl) * Date: 2018-07-09 11:28
> On Fedora after the fix we had also to BuildRequire libnsl2-devel and libtirpc-devel after the glibc change was implemented for the correct headers to be included within the buildroot.

libnsl-devel requires libtirpc-devel here, so that isn't the issue.
History
Date User Action Args
2018-07-09 11:28:17mceplsetmessages: + msg321309
2018-07-06 17:38:14cstrataksetmessages: + msg321186
2018-07-05 17:50:36mceplsetmessages: + msg321121
2018-07-05 11:31:02cstrataksetmessages: + msg321098
2018-07-05 05:14:18mceplsetmessages: + msg321075
2018-07-05 05:08:09mceplsetnosy: + mcepl
messages: + msg321073
2018-02-06 15:55:47cstrataksetstatus: closed -> open

messages: + msg311729
2018-01-27 08:57:00christian.heimessetstatus: open -> closed
type: compile error
resolution: fixed
stage: patch review -> resolved
2018-01-27 08:39:41christian.heimessetmessages: + msg310848
2018-01-27 08:39:18christian.heimessetmessages: + msg310847
2018-01-26 22:36:35christian.heimessetpull_requests: + pull_request5199
2018-01-26 22:32:23christian.heimessetpull_requests: + pull_request5198
2018-01-26 22:28:48christian.heimessetmessages: + msg310813
2018-01-15 11:56:21christian.heimessetmessages: + msg309967
2018-01-15 11:54:33christian.heimessetpull_requests: + pull_request5044
2018-01-12 17:03:38cstrataksetmessages: + msg309863
2018-01-12 15:35:28christian.heimessetmessages: + msg309861
2018-01-12 15:29:39christian.heimessetmessages: + msg309860
2018-01-12 14:27:42python-devsetpull_requests: + pull_request5022
2018-01-12 14:26:41python-devsetstage: patch review
pull_requests: + pull_request5021
2018-01-12 14:26:34christian.heimessetmessages: + msg309856
2018-01-09 05:54:22benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg309684
2018-01-08 19:45:58christian.heimessetassignee: christian.heimes ->
stage: patch review -> (no value)
messages: + msg309681
versions: - Python 3.5, Python 3.8
2018-01-08 17:03:38christian.heimessetkeywords: + patch
stage: patch review
pull_requests: + pull_request4997
2018-01-08 16:52:46christian.heimessetassignee: christian.heimes

nosy: + christian.heimes
2018-01-08 16:48:16vstinnersetmessages: + msg309678
2018-01-08 16:45:20cstrataksetmessages: + msg309677
2018-01-08 16:43:45cstrataksetmessages: + msg309676
2018-01-08 16:41:57vstinnersetmessages: + msg309675
2018-01-08 16:41:17vstinnersetnosy: + vstinner
2018-01-08 16:35:17cstrataksettitle: NIS module fails to build due to the remove of interfaces related to Sun RPC from glibc. -> NIS module fails to build due to the removal of interfaces related to Sun RPC from glibc.
2018-01-08 16:35:03cstratakcreate