Author vstinner
Recipients vstinner
Date 2019-04-18.15:02:15
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1555599736.54.0.585610547316.issue36659@roundup.psfhosted.org>
In-reply-to
Content
Since 2010, the Fedora packages of Python are using a patch on distutils UnixCCompiler to remove standard library path from rpath. The patch has been written by David Malcolm for Python 2.6.4:

* https://src.fedoraproject.org/rpms/python38/blob/master/f/00001-rpath.patch
* https://src.fedoraproject.org/rpms/python2/c/f5df1f834310948b32407933e3b8713e1121105b

I propose to make this change upstream so other Linux distributions will benefit on this change: see attached PR.

"rpath" stands for "run-time search path". Dynamic linking loaders use the rpath to find required libraries:
https://en.wikipedia.org/wiki/Rpath

Full example. Install Python in /opt/py38 with RPATH=/opt/py38/lib, to ensure that Python looks for libpython in this directory:

$ cd path/to/python/sources
$ ./configure --prefix /opt/py38 LDFLAGS="-Wl,-rpath=/opt/py38/lib/" --enable-shared
$ make
$ make install  # on my system, my user can write into /opt ;-)
$ objdump -a -x /opt/py38/bin/python3.8|grep -i rpath
  RPATH                /opt/py38/lib/
$ objdump -a -x /opt/py38/lib/libpython3.8m.so|grep -i rpath
  RPATH                /opt/py38/lib/

Python is installed with RPATH:

$ /opt/py38/bin/python3.8 -m sysconfig|grep -i rpath
	BLDSHARED = "gcc -pthread -shared -Wl,-rpath=/opt/py38/lib/"
	CONFIGURE_LDFLAGS = "-Wl,-rpath=/opt/py38/lib/"
	CONFIG_ARGS = "'--prefix' '/opt/py38' 'LDFLAGS=-Wl,-rpath=/opt/py38/lib/' '--enable-shared'"
	LDFLAGS = "-Wl,-rpath=/opt/py38/lib/"
	LDSHARED = "gcc -pthread -shared -Wl,-rpath=/opt/py38/lib/"
	PY_CORE_LDFLAGS = "-Wl,-rpath=/opt/py38/lib/"
	PY_LDFLAGS = "-Wl,-rpath=/opt/py38/lib/"

Now the difference is how these flags are passed to third party C extensions.

$ cd $HOME
$ /opt/py38/bin/python3.8 -m venv opt_env
$ opt_env/bin/python -m pip install lxml
$ objdump -a -x $(opt_env/bin/python -c 'import lxml.etree; print(lxml.etree.__file__)')|grep -i rpath
  RPATH                /opt/py38/lib/

lxml is compiled with the RPATH. This issue proposes to omit the Python RPATH here.

Comparison with Fedora Python which already contains the change:

$ python3 -m venv fed_venv  # FYI: it's Python 3.7 on Fedora 29
$ fed_venv/bin/python -m pip install lxml
$ objdump -a -x $(fed_venv/bin/python -c 'import lxml.etree; print(lxml.etree.__file__)')|grep -i rpath

^^ empty output: no RPATH, it's the expected behavior

... I'm not sure that the example using /usr/bin/python3.7 is useful, because it's not built using RPATH ...

$ objdump -a -x /usr/bin/python3.7 |grep -i rpath
$ python3.7 -m sysconfig|grep -i rpath
$ objdump -a -x /usr/lib64/libpython3.7m.so |grep -i rpath

^^ no output, it's not built with RPATH
History
Date User Action Args
2019-04-18 15:02:16vstinnersetrecipients: + vstinner
2019-04-18 15:02:16vstinnersetmessageid: <1555599736.54.0.585610547316.issue36659@roundup.psfhosted.org>
2019-04-18 15:02:16vstinnerlinkissue36659 messages
2019-04-18 15:02:15vstinnercreate