classification
Title: --enable-shared links extensions to libpython statically
Type: Stage:
Components: Distutils Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: gustavo, loewis, marienz
Priority: normal Keywords:

Created on 2006-11-22 00:29 by marienz, last changed 2007-02-09 12:38 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
linux-shlib.diff loewis, 2006-11-25 13:42 version 1
Messages (8)
msg30644 - (view) Author: Marien Zwart (marienz) Date: 2006-11-22 00:29
python 2.5 tries to link extension modules to libpython2.5 if python is compiled with --enable-shared (patch #1429775, bug #83279, svn r45232). To do this it adds -lpython2.5 and -L$PREFIX/lib/python2.5/config to the link command line. -lpython2.5 is fine, however the "config" directory it adds contains a static libpython2.5.a library. The libpython2.5.so I think it should be linking to is in $PREFIX/lib. The result is even a trivial extension pulls in (nearly) all of that library, so you get an extension that is over a megabyte in size where older pythons produce one of a few kilobytes.

There is a comment on the referenced bug saying

"""
You can probably rely on libpythonxy.so ending up in
$(DESTDIR)$(LIBDIR)/$(INSTSONAME), whose values you can
retrieve from the installed Makefile (i.e. through
distutils.config).
"""

so I think the patch that got applied does not do what was intended.
msg30645 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-11-22 07:09
I can't reproduce the problem. Why do you think -L$PREFIX/lib/python2.5/config is added to the link command line? AFAICT, it never is.

What operating system are you using?
msg30646 - (view) Author: Marien Zwart (marienz) Date: 2006-11-22 13:02
I can reproduce this by using either gentoo's python 2.5 or one I installed temporarily with ./configure --enable-shared --prefix $HOME/tmp/pytem, using a trivial distutils extension (I used http://dev.gentooexperimental.org/~marienz/ext.c and http://dev.gentooexperimental.org/~marienz/setup.py for testing). The relevant command that is run is:

gcc -pthread -shared build/temp.linux-i686-2.5/ext.o -L/home/marienz/tmp/pytem/lib/python2.5/config -lpython2.5 -o build/lib.linux-i686-2.5/ext.so

for the manually-configured python and something similar for gentoo's python. The code doing the adding was added by r45232 in svn. From the diff (svn di -r45231:45232 http://svn.python.org/projects/python/trunk/Lib/distutils/command/build_ext.py) with some extra context added:

-        if sys.platform[:6] == 'cygwin' or sys.platform[:6] == 'atheos':
+        if sys.platform[:6] == 'cygwin' or sys.platform[:6] == 'atheos' or \
+               (sys.platform.startswith('linux') and
+                sysconfig.get_config_var('Py_ENABLE_SHARED')):
             if string.find(sys.executable, sys.exec_prefix) != -1:
                 # building third party extensions
                 self.library_dirs.append(os.path.join(sys.prefix, "lib",
                                                      "python" + get_python_version(),
                                                      "config"))

(that is around line 188 of Lib/distutils/command/build_ext.py)

sys.platform on this host is linux2 and as far as I can tell Py_ENABLE_SHARED is true if --enable-shared is passed to configure.

If you need any more information please ask.
msg30647 - (view) Author: Gustavo J. A. M. Carneiro (gustavo) * Date: 2006-11-23 20:47
Hmm.. I think I wrongfully assumed $prefix/lib/pythonX.Y/config contained a shared python library in addition to the static one.  It seems that was an Ubuntu specific thing :|

I'll take a good look and fix this within a couple of days...
msg30648 - (view) Author: Gustavo J. A. M. Carneiro (gustavo) * Date: 2006-11-25 13:10
*sigh* why doesn't SF let me attach patches? :(

You can find a patch to fix this here: http://www.gnome.org/~gjc/linux-shlib.diff
msg30649 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-11-25 13:42
gustavo: you can only attach a patch if you are team member or creator of the bug report. I'm attaching your patch.

marienz: can you please confirm whether this patch solves this problem?
msg30650 - (view) Author: Marien Zwart (marienz) Date: 2006-11-27 01:01
Yes, that seems to fix it, thanks!
msg30651 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-02-09 12:38
Thanks for the patch. Committed as r53691 and r53692.
History
Date User Action Args
2006-11-22 00:29:27marienzcreate