classification
Title: getpath.c doesn't know about lib64
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder: Add sys.platlibdir and configure --with-platlibdir to use /usr/lib64 on Fedora and SuSE
View: 1294959
Assigned To: Nosy List: lemburg, pflagerd, vstinner
Priority: normal Keywords:

Created on 2020-05-04 21:51 by lemburg, last changed 2020-12-14 08:51 by vstinner. This issue is now closed.

Messages (6)
msg368086 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2020-05-04 21:51
On platforms which configure identifies as bi-arch platform, libdir is set to $[exec_prefix}/lib64, which results in the C extensions to get installed in e.g. /usr/local/lib64/python3.8/lib-dynload/.

However, the getpath.c routines use a fixed "lib/python" VERSION (see https://github.com/python/cpython/blob/3.8/Modules/getpath.c#L1200) path to build sys.path. As a result, the built Python binary cannot load the builtin C extensions.

A work-around on OpenSUSE is to set CONFIG_SITE="" when configuring Python. This disables the bi-arch support and has libdir default to ${exec_prefix}/lib again.

Looking at the master branch, this may already have been fixed for 3.9, since a PLATLIBDIR variable is used instead. The patch would have to be backported to earlier Python versions as well.
msg368087 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2020-05-04 21:53
Just to clarify: the CONFIG_SITE script on OpenSUSE causes configure to use lib64, not the Python configure script itself.
msg368088 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-04 21:54
Hum, this issue seems to be specific to Fedora and SuSE. Did you open an issue on OpenSuSE bug tracker?

> Looking at the master branch, this may already have been fixed for 3.9, since a PLATLIBDIR variable is used instead. The patch would have to be backported to earlier Python versions as well.

Yeah, this issue looks like a duplicate of bpo-1294959 which has been fixed. I suggest to close the issue. I had to a new attribute to the sys module, not like the thing that we can do in a stable version.
msg368092 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2020-05-04 22:15
No, I have not opened a bug report on OpenSUSE. Since the OS uses bi-arch throughout, using lib64 is the natural thing to use for libdir on the OS.

I think the issue lies with getpath.c only, since it makes an assumption about the libdir config value, which doesn't necessarily hold in practice. The libdir config value can be changed via the standard --libdir configure parameter, so getpath.c should really not assume that the default setting is always used.

Having a variable in Python 3.9 is nice, but perhaps we can still make Python 3.7 and 3.8 work as well.

The correct approach to building the full lib_python path is not to use exec_prefix + "/lib/python" + VERSION, but instead to use libdir + "/python" + VERSION (which corresponds to the Makefile variable BINLIBDEST). libdir would have to be taken from the config variable LIBDIR.
msg382928 - (view) Author: Daniel Pflager (pflagerd) Date: 2020-12-13 03:04
Python-3.9.1 OpenSUSE 15.1

While ./configure --with-platlibdir=lib64 does work-around the problem on OpenSUSE by putting everything into lib64/, this is not the default behavior. In other words, should Python-3.9.1 be built on OpenSUSE 15.1 without --with-platlibdir=lib64, the behavior described in this bug report occurs.

Not everyone building Python-3.9.1 on OpenSUSE will take the time to research this and end up finding https://bugs.python.org/issue1294959 (as I did after considerable effort).

Intuitively, doesn't one expect configure to work this kind of thing out (and if it can't, warn the user)?
msg382963 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-14 08:51
This issue is a duplicate of bpo-1294959.
History
Date User Action Args
2020-12-14 08:51:39vstinnersetstatus: open -> closed
superseder: Add sys.platlibdir and configure --with-platlibdir to use /usr/lib64 on Fedora and SuSE
messages: + msg382963

resolution: fixed
stage: resolved
2020-12-13 03:04:12pflagerdsetnosy: + pflagerd
messages: + msg382928
2020-05-04 22:15:13lemburgsetmessages: + msg368092
2020-05-04 21:54:12vstinnersetnosy: + vstinner
messages: + msg368088
2020-05-04 21:53:19lemburgsetmessages: + msg368087
2020-05-04 21:51:07lemburgcreate