classification
Title: libpython3.so: Broken soname and linking
Type: Stage: resolved
Components: Build Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: loewis Nosy List: Arfrever, barry, dmalcolm, georg.brandl, loewis, python-dev
Priority: release blocker Keywords: patch

Created on 2011-02-27 20:26 by Arfrever, last changed 2011-05-09 05:43 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
libpython3.so.patch Arfrever, 2011-05-07 21:46 review
Messages (10)
msg129649 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-02-27 20:26
1. libpython3.so has soname "llibpython3.so" due to needless "l" in "-Wl,-hl$@".
2. libpython3.so is not linked against libpython$(LDVERSION).so when -Wl,--as-needed flag is used. -Wl,--as-needed flag is used by default in Gentoo due to major benefits.
From `info ld`:
`--as-needed'
`--no-as-needed'
     This option affects ELF DT_NEEDED tags for dynamic libraries
     mentioned on the command line after the `--as-needed' option.
     Normally the linker will add a DT_NEEDED tag for each dynamic
     library mentioned on the command line, regardless of whether the
     library is actually needed or not.  `--as-needed' causes a
     DT_NEEDED tag to only be emitted for a library that satisfies an
     undefined symbol reference from a regular object file or, if the
     library is not found in the DT_NEEDED lists of other libraries
     linked up to that point, an undefined symbol reference from
     another dynamic library.  `--no-as-needed' restores the default
     behaviour.

I'm attaching the patch.
msg129652 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-02-27 20:52
Regardless of this patch, such a libpython3.so regular file cannot exist in Gentoo. Gentoo package manager disallows installation, when it detects file collisions. If I rename libpython3.so of Python 3.2 to libpython3-3.2.so and libpython3.so of Python 3.3 to libpython3-3.3.so, then ldconfig will create libpython3.so symlink pointing to potentially wrong target library. This problem can be avoided by removing soname from libpython3.so, so whole -Wl,-h${soname} could be removed.
msg129682 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-28 06:31
Having the soname be libpython3 is the whole point of the library, it serves no other reason.

It is intentional that there are file collisions with that file, and either the local admin or the distributor must make an explicit choice which libpython3 should be installed; it should be the one that corresponds to /usr/bin/python (if you install it into /usr/lib).

As for file20939: it's GNU-ld-specific, so it likely breaks if the linker is not GNU ld.
msg129683 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-02-28 06:41
The -hl -> -h part seems correct though.
msg129723 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-28 21:56
> The -hl -> -h part seems correct though.

Indeed.
msg129724 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-02-28 21:57
"Having the soname be libpython3 is the whole point of the library": I think that the point of this library is "libpython3.so" filename, not soname. Everything should still work if libpython3.so had no soname at all.

(If somebody has installed unpatched Python 3.2, then ldconfig will create llibpython3.so symlink (e.g. /usr/lib/llibpython3.so) and linker will hardcode dependency on llibpython3.so in generated libraries/executables during linking against libpython3.so (e.g. when -lpython3 has been specified).)

(I will work on improved patch for -Wl,--no-as-needed.)
msg129725 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-28 22:04
> "Having the soname be libpython3 is the whole point of the library":
> I think that the point of this library is "libpython3.so" filename,
> not soname. Everything should still work if libpython3.so had no
> soname at all.

IIUC, the linker will hard-code the full path name (include directory
name) into DT_NEEDED if the shared library has no soname. That would
be undesirable - the DT_NEEDED reference must be just the file name,
not the full path.
msg135506 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-05-07 21:46
I'm attaching improved patch, which adds detection of -Wl,--no-as-needed flag. This issue should be release blocker for Python 3.2.1.
msg135517 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-05-08 05:00
Martin?
msg135564 - (view) Author: Roundup Robot (python-dev) Date: 2011-05-09 05:43
New changeset 749686064642 by Martin v. Löwis in branch '3.2':
Use --as-needed when linking libpython3.so. Closes #11347.
http://hg.python.org/cpython/rev/749686064642
History
Date User Action Args
2011-05-09 05:43:37python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg135564

resolution: fixed
stage: resolved
2011-05-08 05:00:55georg.brandlsetpriority: normal -> release blocker
2011-05-08 05:00:37georg.brandlsetmessages: + msg135517
2011-05-07 21:46:20Arfreversetfiles: + libpython3.so.patch

messages: + msg135506
2011-05-07 21:34:43Arfreversetfiles: - libpython3.so.patch
2011-02-28 22:04:52loewissetnosy: loewis, barry, georg.brandl, Arfrever, dmalcolm
messages: + msg129725
2011-02-28 21:57:50Arfreversetnosy: loewis, barry, georg.brandl, Arfrever, dmalcolm
messages: + msg129724
2011-02-28 21:56:37loewissetnosy: loewis, barry, georg.brandl, Arfrever, dmalcolm
messages: + msg129723
2011-02-28 06:41:38georg.brandlsetnosy: + georg.brandl
messages: + msg129683
2011-02-28 06:31:36loewissetnosy: loewis, barry, Arfrever, dmalcolm
messages: + msg129682
2011-02-28 01:29:14pitrousetnosy: + barry, dmalcolm
2011-02-27 20:52:37Arfreversetmessages: + msg129652
2011-02-27 20:28:48georg.brandlsetassignee: loewis
2011-02-27 20:26:27Arfrevercreate