Author vstinner
Recipients doko, vstinner
Date 2019-04-25.18:25:47
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1556216748.62.0.992604554579.issue36721@roundup.psfhosted.org>
In-reply-to
Content
The bpo-21536 modified how C extensions are built: they are no longer linked to libpython. The problem is that when an application embeds Python: the application wants to be linked to libpython.

commit 8c3ecc6bacc8d0cd534f2b5b53ed962dd1368c7b (HEAD -> master, upstream/master)
Author: Victor Stinner <vstinner@redhat.com>
Date:   Thu Apr 25 20:13:10 2019 +0200

    bpo-21536: C extensions are no longer linked to libpython (GH-12946)
    
    On Unix, C extensions are no longer linked to libpython.
    
    It is now possible to load a C extension built using a shared library
    Python with a statically linked Python.
    
    When Python is embedded, libpython must not be loaded with
    RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
    was already not possible to load C extensions which were not linked
    to libpython, like C extensions of the standard library built by the
    "*shared*" section of Modules/Setup.
    
    distutils, python-config and python-config.py have been modified.

I chose to modify distutils, python-config (shell) and python-config.py (Python), but *not* Misc/python.pc (pkg-config).

Previously, we had:

$ pkg-config python-3.7 --libs
-lpython3.7m 
$ python3.7-config --libs
-lpython3.7m -lcrypt -lpthread -ldl -lutil -lm 
$ python3.7-config.py --libs
-lpython3.7m -lcrypt -lpthread -ldl -lutil -lm

Now, we get:

$ pkg-config python-3.8 --libs
-lpython3.8
$ python3.8-config --libs
-lcrypt -lpthread -ldl -lutil -lm -lm 
$ python-config.py --libs
-lcrypt -lpthread -ldl -lutil -lm -lm

python-config and python-config.py can now be used to build C extensions, but not to build an application embedding Python.

pkg-config should not be used to build a C extenstion since it links to libpython, but we don't want to do that (see bpo-21536).

I'm not sure that different tools should return different results.

I propose:

* Add a new command "pkg-config python-3.8-embed"
* Add a new "--embed" option to python3.8-config and python3.8-config.py commands
* Remove "-lpython@VERSION@@ABIFLAGS@" from "Libs: -L${libdir} -lpython@VERSION@@ABIFLAGS@" of Misc/python.pc.in

On Windows, we already provide different binaries for embedded Python with "-embed" suffix:

* Download Windows x86-64 embeddable zip file: python-3.7.3-embed-amd64.zip
* Download Windows x86-64 executable installer: python-3.7.3-amd64.exe

https://www.python.org/downloads/windows/
History
Date User Action Args
2019-04-25 18:25:48vstinnersetrecipients: + vstinner, doko
2019-04-25 18:25:48vstinnersetmessageid: <1556216748.62.0.992604554579.issue36721@roundup.psfhosted.org>
2019-04-25 18:25:48vstinnerlinkissue36721 messages
2019-04-25 18:25:47vstinnercreate