classification
Title: --libs is inconsistent for python-config --libs and pkgconfig python --libs
Type: behavior Stage:
Components: Build Versions: Python 3.5, Python 3.4, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, doko, haubi, ned.deily, tdsmith
Priority: normal Keywords: needs review, patch

Created on 2012-08-08 10:43 by doko, last changed 2017-03-20 10:07 by haubi.

Files
File name Uploaded Description Edit
32143cda4d80.diff haubi, 2014-06-04 16:25 proposed fix, although also containing diff for issue#21660 review
Pull Requests
URL Status Linked Edit
PR 737 open haubi, 2017-03-20 10:07
Repositories containing patches
http://hg.code.sf.net/p/prefix-launcher/cpython#issue15590
Messages (11)
msg167683 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2012-08-08 10:43
seen with at least 2.7, 3.2 and 3.3:

$ python-config --libs
-lpthread -ldl -lutil -lm -lpython2.7

$ pkg-config python --static --libs
-lpthread -ldl -lutil -lpython2.7

python-config uses the SYSLIBS macro, while pkg-config uses the LIBS macro.

depending on what you want to do, both can be wrong or right:

 - to build an extension, LIBS should be used.

 - to link an embedded interpreter, SYSLIBS should be used,
   which might not be complete.
   Howver in this case MODLIBS is missing.

a patch should be easy once we can define the intended behaviour.
msg167686 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2012-08-08 11:11
Behavior of 'pkg-config python' is more useful since without --static option only the set of directly needed libraries is returned.

In Python 3.3 or 3.4 --static option could be introduced in python-config and behavior without this option could be changed to match 'pkg-config python'.
msg219665 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-03 06:33
Well, they should not be identical, as they are for different use cases.

"pkg-config python" is to build an application containing a python interpreter (like python$EXE):
 + Link against libpython.so. Additionally,
 + re-export symbols from libpython.so for the python-modules (platform-specific).
 = This is similar to build against any other library, thus using 'pkg-config python'.

"python-config" is to build a python-module (like build/lib.<platform>-<pyver>/*.so):
 + No need to link against libpython.so, instead
 + expect symbols from libpython.so to be available at runtime, platform specific either
 + as a list of symbols to import from "the main executable" (AIX), or
 + as undefined symbols at build-time (Linux, others), or
 = This is specific to python-modules, thus using 'python-config'.
msg219679 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-06-03 10:58
They are not for different cases. Difference between is a result of the fact that they were implemented independently at different times.

Linking of extension modules against libpythonX.Y.so is a good idea anyway. It avoids build failure with LDFLAGS="-Wl,--no-undefined".
msg219680 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-06-03 11:06
they are.  assume you build the zlib and elementtree extensions as builtins, then libs for an interpreter includes libz and libexpat, while they are not needed for extensions.
msg219764 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-04 16:24
So this diff - depending on issue#21660 - now drops showing any $LIBS from python-config, as python-modules usually do not link against any python-known libraries.

Instead, now there is a new configure variable LINKFORMODULE, which is shown by python-config --ldflags.

And $LINKFORSHARED is added to python.pc (for pkg-config).

Eventually, this allows to drop those Darwin hackery wrt. python-framework, as the python library isn't linked against the modules any more - but to PYTHONFRAMEWORK via LINKFORMODULE instead.

However, I don't have any Darwin around here: anyone?

For AIX, this looks good so far.

Thanks!
msg219774 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2014-06-04 22:28
Michael Haubenwallner: Could you show output of 'python-config-3.5 --ldflags' and 'python-config-3.5 --libs' without and with this patch (on GNU/Linux and AIX)?
msg219821 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-05 15:54
For AIX, with both these configure variants:
  $ configure --prefix=/prefix --enable-shared CC=gcc CXX=g++ OPT=
  $ configure --prefix=/prefix --enable-shared --without-computed-gotos CC=xlc_r CXX=xlC_r OPT=
the output changes like this:

   $ PKG_CONFIG_PATH=/prefix/lib/pkgconfig pkg-config --libs python-3.4
old: -L/prefix/lib -lpython3.4m
new: -L/prefix/lib -lpython3.4m -Wl,-bE:/prefix/lib/python3.4/config-3.4m/python.exp -lld

   $ /prefix/bin/python3.4-config --ldflags
old:  -L/prefix/lib -lintl -ldl -lm  -lpython3.4m -Wl,-bE:Modules/python.exp -lld
new: -Wl,-bI:/prefix/lib/python3.4/config-3.4m/python.exp



For Linux, with this configure variant:
  $ configure --prefix=/prefix --enable-shared CC=gcc CXX=g++
the output changes like this:

  $ PKG_CONFIG_PATH=/prefix/lib/pkgconfig pkg-config --libs python-3.4
old: -L/prefix/lib -lpython3.4m
new: -L/prefix/lib -lpython3.4m -Xlinker -export-dynamic

   $ /prefix/bin/python3.4-config --ldflags
old:   -L/prefix/lib -lpthread -ldl  -lutil -lm  -lpython3.4m -Xlinker -export-dynamic
new: # Yes, nothing. Not sure if python modules should have undefined python symbols, or link against libpython.so

Just found out that distutils.command.build_ext.get_libraries() does add libpython.so for Linux and similar (not Darwin and AIX).
Actually, distutils.command.build_ext.get_libraries() should add $LINKFORMODULE instead now, so LDSHARED does not need to carry.

More thoughts?  (will post results with --disable-shared as separate comment)
msg219823 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-05 16:24
Now for --disable-shared:

For AIX, with both these configure variants:
  $ configure --prefix=/prefix --disable-shared CC=gcc CXX=g++ OPT=
  $ configure --prefix=/prefix --disable-shared --without-computed-gotos CC=xlc_r CXX=xlC_r OPT=
the output changes like this:

   $ PKG_CONFIG_PATH=/prefix/lib/pkgconfig pkg-config --libs python-3.4
old: -L/prefix/lib -lpython3.4m
new: -L/prefix/lib -lpython3.4m -Wl,-bE:/prefix/lib/python3.4/config-3.4m/python.exp -lld

   $ /prefix/bin/python3.4-config --ldflags
old: -L/prefix/lib/python3.4/config-3.4m -L/prefix/lib -lintl -ldl -lm  -lpython3.4m -Wl,-bE:Modules/python.exp -lld
new: -Wl,-bI:/prefix/lib/python3.4/config-3.4m/python.exp



For Linux, with this configure variant:
  $ configure --prefix=/prefix --enable-shared CC=gcc CXX=g++
the output changes like this:

  $ PKG_CONFIG_PATH=/prefix/lib/pkgconfig pkg-config --libs python-3.4
old: -L/prefix/lib -lpython3.4m
new: -L/prefix/lib -lpython3.4m -Xlinker -export-dynamic

   $ /prefix/bin/python3.4-config --ldflags
old: -L/prefix/lib/python3.4/config-3.4m -L/prefix/lib -lpthread -ldl  -lutil -lm  -lpython3.4m -Xlinker -export-dynamic
new: # Yes, nothing. 

And with --disable-shared, even distutils.command.build_ext.get_libraries() does *not* add libpython.so for Linux and similar, so python-shipped modules are linked with python symbols undefined now - but still as shared libraries.
msg219825 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-05 16:37
Erm, the latter should read:
  For Linux, with this configure variant:
    $ configure --prefix=/prefix --disable-shared CC=gcc CXX=g++


Now reading GNU ld manpage for Linux:

>   $ PKG_CONFIG_PATH=/prefix/lib/pkgconfig pkg-config --libs python-3.4
> new: -L/prefix/lib -lpython3.4m -Xlinker -export-dynamic

ld(1), at '--export-dynamic', tells about '--dynamic-list=file' to export specific symbols only - which actually looks similar to '-bE:file' for AIX...
msg235875 - (view) Author: Tim Smith (tdsmith) * Date: 2015-02-13 05:00
On Darwin, it would be nice if LINKFORMODULE used "-undefined dynamic_lookup" instead of explicitly linking to a framework binary. Modules with explicit links to a framework cause segfaults when they are imported from a different, but compatible, framework interpreter -- i.e., building against the system Python 2.7 but importing from a user-installed Python 2.7 causes an immediate crash.

I'm a maintainer of Homebrew, a package manager for OS X, and I spend a lot of time tracking down and eliminating errant explicit framework links because they make packaging binaries much harder for us and cause crashes for users.
History
Date User Action Args
2017-03-20 10:07:17haubisetpull_requests: + pull_request652
2015-02-13 05:00:37tdsmithsetnosy: + tdsmith
messages: + msg235875
2014-06-05 16:37:19haubisetmessages: + msg219825
2014-06-05 16:24:46haubisetmessages: + msg219823
2014-06-05 15:54:26haubisetmessages: + msg219821
2014-06-04 22:28:23Arfreversetmessages: + msg219774
2014-06-04 19:41:47ned.deilysetnosy: + ned.deily
2014-06-04 16:25:23haubisetfiles: + 32143cda4d80.diff
keywords: + patch
2014-06-04 16:25:00haubisethgrepos: + hgrepo250
messages: + msg219764
2014-06-03 11:06:26dokosetmessages: + msg219680
2014-06-03 10:58:54Arfreversetmessages: + msg219679
2014-06-03 06:33:18haubisetnosy: + haubi
messages: + msg219665
2014-06-02 21:39:39ned.deilysetversions: + Python 2.7, Python 3.4, Python 3.5, - Python 3.3
2012-08-08 11:11:54Arfreversetnosy: + Arfrever
messages: + msg167686
2012-08-08 10:43:17dokocreate