classification
Title: cannot find -lpythonX.X when building Python on FreeBSD with --enable-shared
Type: compile error Stage: resolved
Components: Build, Distutils, Library (Lib) Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Arfrever, akitada, christian.heimes, eric.araujo, koobs, loewis, nbastin, pitrou, python-dev, tarek
Priority: normal Keywords: patch

Created on 2008-11-20 17:26 by akitada, last changed 2013-09-28 23:55 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
4366.diff nbastin, 2010-01-09 17:56 review
python-issue4366.diff koobs, 2013-09-22 13:49 review
Messages (19)
msg76117 - (view) Author: Akira Kitada (akitada) * Date: 2008-11-20 17:26
I get a number of "cannot find -lpython2.5" error when building Python
2.5.2 on FreeBSD 2.5.2 with gcc 2.95.4.
This problem is only occured when I build it with "--enable-shared"
configure option.

This is how you can reproduce this problem.

cd Python-2.5.2
configure --enable-shared
make

and you will get

gcc -shared
build/temp.freebsd-4.11-RELEASE-i386-2.5/usr/home/build/dev/Python-2.5.2/Modules/_struct.o
-L/usr/local/lib -lpython2.5 -o build/lib.freebsd-4.11-RELEASE-i386-2.5/_
struct.so
/usr/libexec/elf/ld: cannot find -lpython2.5
...
/home/build/dev/Python-2.5.2/Modules/_ctypes/libffi/src/x86/sysv.o
-L/usr/local/lib -lpython2.5 -o
build/lib.freebsd-4.11-RELEASE-i386-2.5/_ctypes.so
/usr/libexec/elf/ld: cannot find -lpython2.5
...
gcc -shared
build/temp.freebsd-4.11-RELEASE-i386-2.5/usr/home/build/dev/Python-2.5.2/Modules/_ctypes/_ctypes_test.o
-L/usr/local/lib -lpython2.5 -o build/lib.freebsd-4.11-RELEA
SE-i386-2.5/_ctypes_test.so
/usr/libexec/elf/ld: cannot find -lpython2.5 
...
gcc -shared
build/temp.freebsd-4.11-RELEASE-i386-2.5/usr/home/build/dev/Python-2.5.2/Modules/_weakref.o
-L/usr/local/lib -lpython2.5 -o build/lib.freebsd-4.11-RELEASE-i386-2.5/
_weakref.so
/usr/libexec/elf/ld: cannot find -lpython2.5
...
gcc -shared
build/temp.freebsd-4.11-RELEASE-i386-2.5/usr/home/build/dev/Python-2.5.2/Modules/arraymodule.o
-L/usr/local/lib -lpython2.5 -o build/lib.freebsd-4.11-RELEASE-i386-2
.5/array.so
/usr/libexec/elf/ld: cannot find -lpython2.5
......


You can workaround this by running  ./configure LDFLAGS="-L."
--enable-shared.
msg76118 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-11-20 17:33
Please try this patch with a clean source tree. It adds the current
directory to the library search path.

Index: setup.py
===================================================================
--- setup.py    (revision 67295)
+++ setup.py    (working copy)
@@ -245,6 +245,7 @@
     def detect_modules(self):
         # Ensure that /usr/local is always used
         add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
+        add_dir_to_list(self.compiler.library_dirs, '.')
         add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')

         # Add paths specified in the environment variables LDFLAGS and
msg76122 - (view) Author: Akira Kitada (akitada) * Date: 2008-11-20 17:54
Christian's patch fixed this problem!
(tested on 4.11-RELEASE)

I'm not sure why the other platforms don't suffer this problem.
msg76127 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-20 18:44
Since r53691, and issue 1600860, "." is added to library_dirs on Linux
and GNU systems. This probably should be extended to FreeBSD, and other
systems.

The critical point to notice is that the -L option is not only while
building Python itself, but also for extension modules (assuming Python
is installed into a non-standard location).

Now, the question is how to extend this approach to FreeBSD. For 2.5 and
2.6, I think it is safest to explicitly add freebsd to the list of
systems tested for.

For the trunk, and probably 3.0, I would try to add the library whenever
Py_ENABLE_SHARED is defined, and os.name is posix.
msg76361 - (view) Author: Akira Kitada (akitada) * Date: 2008-11-24 21:07
Changing the title because this is not 2.5.x specific.
msg76364 - (view) Author: Akira Kitada (akitada) * Date: 2008-11-24 21:18
Changing the title again because this problem is not FreeBSD 4 specific.
Build on recent FreeBSD also has the same problem.
(I tested this on 6.3, too)
msg76448 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-26 06:47
2.5.3 is out of scope for this issue (and thus, the whole of 2.5). There
is no workable patch, yet, and 2.5.3 is just two weeks ahead.
msg76458 - (view) Author: Akira Kitada (akitada) * Date: 2008-11-26 10:42
Martin,
Two questions:

1. Isn't Christian's patch enough for this?
2. How about Python 3.0 and 2.6.1? Are they also out of scope for this?
msg76480 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-11-26 18:52
> 1. Isn't Christian's patch enough for this?

No. It should be dealt with in the same way as on Linux (or the Linux
way should be changed).

> 2. How about Python 3.0 and 2.6.1? Are they also out of scope for this?

Not yet, no. However, they are soon to be released, so chances are low
that a patch arrives in time.
msg97465 - (view) Author: Nick Bastin (nbastin) * (Python committer) Date: 2010-01-09 17:56
A more appropriate patch should be (for 2.7 trunk - I'm grabbing a checkout of 3.2 trunk now):

Index: build_ext.py
===================================================================
--- build_ext.py	(revision 77388)
+++ build_ext.py	(working copy)
@@ -280,7 +280,7 @@
         # Python's library directory must be appended to library_dirs
         sysconfig.get_config_var('Py_ENABLE_SHARED')
         if ((sys.platform.startswith('linux') or sys.platform.startswith('gnu')
-             or sys.platform.startswith('sunos'))
+             or sys.platform.startswith('sunos') or sys.platform.startswith('freebsd'))
             and sysconfig.get_config_var('Py_ENABLE_SHARED')):
             if sys.executable.startswith(os.path.join(sys.exec_prefix, "bin")):
                 # building third party extensions


I'm hoping someone will weigh in on whether this should be done on all versions of FreeBSD (I don't see why not, but perhaps there is some magic that I don't understand in newer versions).
msg97466 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-01-09 18:21
configure.in has the same action for NetBSD*|FreeBSD*|DragonFly*, so I think distutils should parallel that. Not sure what sys.platform would be on the other BSDs, though.
msg97477 - (view) Author: Nick Bastin (nbastin) * (Python committer) Date: 2010-01-09 23:22
NetBSD is netbsd* and DragonFly is dragonfly* (currently dragonfly2, although I suspect in this way dragonfly1 was identical, if it ever existed).
msg198013 - (view) Author: Kubilay Kocak (koobs) Date: 2013-09-18 10:19
Confirming identical failures on all branches:

tip: /usr/bin/ld: cannot find -lpython3.4m
3.3: /usr/bin/ld: cannot find -lpython3.3m
3.2: /usr/bin/ld: cannot find -lpython3.2m
3.1: /usr/bin/ld: cannot find -lpython3.1
2.7: /usr/bin/ld: cannot find -lpython2.7
2.6: /usr/bin/ld: cannot find -lpython2.6

Tested attached patch with success, will carry this patch locally in all FreeBSD Python ports until it merged. Let's close this 5 year old nasty :)
msg198017 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-18 12:32
Why special-case FreeBSD in the patch? Shouldn't this be done for nearly all Unix systems? (or, at the very least or BSD-likes)?
msg198020 - (view) Author: Kubilay Kocak (koobs) Date: 2013-09-18 12:48
Concur, and then also, why special case linux, gnu and sunos?

The comment is:

# Python's library directory [[must]] be appended to library_dirs (emphasis mine)

Is the real question then "in what cases should the library path NOT be added?" ?

I also note Christian used a different approach (#msg76118), electing to unconditionally add the path in setup.py
msg198277 - (view) Author: Kubilay Kocak (koobs) Date: 2013-09-22 13:49
Attaching an initial patch with the following changes:

- Update comment
- Add original (#1600860) and current issue ID references
- Remove sys.platform conditional
- Remove NOOP sysconfig.get_config_var call (Reported by: birkenfeld)

This results in all platforms receiving the same treatment as the default, which is consistent with the comment
msg198340 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-23 20:37
Koobs, have you signed a contributor's agreement?
See http://www.python.org/psf/contrib/
msg198557 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-28 23:51
New changeset 48d28de5bdf8 by Antoine Pitrou in branch '3.3':
Issue #4366: Fix building extensions on all platforms when --enable-shared is used.
http://hg.python.org/cpython/rev/48d28de5bdf8

New changeset d6e35146ae53 by Antoine Pitrou in branch 'default':
Issue #4366: Fix building extensions on all platforms when --enable-shared is used.
http://hg.python.org/cpython/rev/d6e35146ae53

New changeset 28e6c23c8ae6 by Antoine Pitrou in branch '2.7':
Issue #4366: Fix building extensions on all platforms when --enable-shared is used.
http://hg.python.org/cpython/rev/28e6c23c8ae6
msg198558 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-09-28 23:55
Ok, let's mark this bug fixed.
History
Date User Action Args
2014-02-03 20:11:36Arfreverlinkissue7538 superseder
2013-09-28 23:55:36pitrousetstatus: open -> closed
resolution: fixed
messages: + msg198558

stage: patch review -> resolved
2013-09-28 23:51:43python-devsetnosy: + python-dev
messages: + msg198557
2013-09-23 20:37:56pitrousetmessages: + msg198340
2013-09-22 13:49:12koobssetfiles: + python-issue4366.diff

messages: + msg198277
2013-09-19 01:13:17Arfreversetnosy: + Arfrever
2013-09-18 12:48:49koobssetmessages: + msg198020
2013-09-18 12:32:53pitrousetversions: - Python 2.6, Python 3.1, Python 3.2
nosy: + pitrou

messages: + msg198017

components: + Library (Lib)
stage: patch review
2013-09-18 10:19:23koobssetassignee: eric.araujo

components: + Distutils
versions: + Python 2.6, Python 3.3, Python 3.4
nosy: + eric.araujo, koobs, tarek
title: cannot find -lpythonX.X when buinding Python on FreeBSD -> cannot find -lpythonX.X when building Python on FreeBSD with --enable-shared
messages: + msg198013
2010-08-04 03:09:26terry.reedysetversions: + Python 3.2, - Python 2.6, Python 2.5, Python 3.0
2010-01-09 23:22:41nbastinsetmessages: + msg97477
2010-01-09 18:21:03loewissetmessages: + msg97466
2010-01-09 17:56:30nbastinsetfiles: + 4366.diff

nosy: + nbastin
messages: + msg97465

keywords: + patch
2008-11-26 18:52:22loewissetmessages: + msg76480
2008-11-26 10:42:09akitadasetmessages: + msg76458
2008-11-26 06:47:21loewissetmessages: + msg76448
versions: - Python 2.5.3
2008-11-25 23:41:59akitadasetversions: + Python 2.6, Python 3.0, Python 3.1, Python 2.7, Python 2.5.3
2008-11-24 21:18:18akitadasettitle: cannot find -lpython2.X when buinding Python on FreeBSD 4.11 -> cannot find -lpythonX.X when buinding Python on FreeBSD
messages: + msg76364
versions: - Python 2.6, Python 3.0, Python 3.1, Python 2.7, Python 2.5.3
2008-11-24 21:07:13akitadasettitle: cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4.11 -> cannot find -lpython2.X when buinding Python on FreeBSD 4.11
messages: + msg76361
versions: + Python 2.6, Python 3.0, Python 3.1, Python 2.7, Python 2.5.3
2008-11-20 18:44:13loewissetnosy: + loewis
messages: + msg76127
2008-11-20 17:54:35akitadasetmessages: + msg76122
2008-11-20 17:33:44christian.heimessetnosy: + christian.heimes
messages: + msg76118
2008-11-20 17:26:58akitadacreate