Title: BSD version of ctypes.util.find_library
Type: Stage:
Components: Library (Lib) Versions: Python 2.5
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: theller Nosy List: chmod007, loewis, mkam, nnorwitz, theller
Priority: critical Keywords: patch

Created on 2006-12-07 13:29 by mkam, last changed 2007-01-17 19:59 by theller. This issue is now closed.

File name Uploaded Description Edit mkam, 2006-12-07 13:29 mkam, 2006-12-12 11:28 improved patch
Messages (14)
msg51492 - (view) Author: Martin Kammerhofer (mkam) Date: 2006-12-07 13:29
The ctypes.util.find_library function for Posix systems is actually
tailored for Linux systems. While the _findlib_gcc function relies
only on the GNU compiler and may therefore work on any system with the
"gcc" command in PATH, the _findLib_ld function relies on the
/sbin/ldconfig command (originating from SunOS 4.0) which is not
standardized. The version from GNU libc differs in option syntax and
output format from other ldconfig programs around.

I therefore provide a patch that enables find_library to properly
communicate with the ldconfig program on FreeBSD systems. It has been
tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD
systems too. (It works without this patch on FreeBSD, because after
getting an error from ldconfig it falls back to _findlib_gcc.)

While at it I also tidied up the Linux specific code: I'm escaping the
function argument before interpolating it into a regular expression (to
protect against nasty regexps) and removed the code for creation of a
temporary file that was not used in any way.
msg51493 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-12-07 19:15
Thomas, can you take a look?
msg51494 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2006-12-07 21:11
Will do (although I would appreciate review from others too; I'm not exactly a BSD expert).
msg51495 - (view) Author: David Remahl (chmod007) Date: 2006-12-08 07:50
# Does this work (without the gcc fallback) on other *BSD systems too?

I don't know, but it doesn't work on Darwin (which already has a custom method through macholib).
msg51496 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2006-12-08 20:32
I have tested the patch on FreeBSD 6.0 and (after extending the check to test for sys.platform.startswith("openbsd")) on OpenBSD 3.9 and it works fine.

find_library("c") now returns on FreeBSD 6.0, and in OpenBSD 3.9, while it returned 'None' before on both machines.
msg51497 - (view) Author: Martin Kammerhofer (mkam) Date: 2006-12-11 10:10
Hm, I did not know that OpenBSD is still using two version numbers for shared library.
(I conclude that from the "" in the previous followup. Btw FreeBSD has used
a MAJOR.MINOR[.DEWEY] scheme during the ancient days of the aout executable format.)
Unfortunately my freebsd patch has the assumption of a single version number built in;
more specifically the
  cmp(* map(lambda x: int(x.split('.')[-1]), (a, b)))
is supposed to sort based an the last dot separated field. I guess that OpenBSD system
does not have another libc, at least none with a minor > 0. ;-)
Thomas, can you mail me the output of "ldconfig -r"? I will refine the patch then,
doing a more general sort algorithm; i.e. sort by all trailing /(\.\d+)+/ fields. Said output from NetBSD welcome too. DragonflyBSD should be no problem since it is a fork of FreeBSD 4.8, but what looks its sys.platform like?
msg51498 - (view) Author: Martin Kammerhofer (mkam) Date: 2006-12-12 11:28
Here is the revised patch. Tested on a (virtual) OpenBSD 3.9 machine, FreeBSD 4.11, FreeBSD 6.2 and DragonFlyBSD 1.6. Does not make assumptions on how many version numbers are appended to a library name any more. Even mixed length names (e.g. vs. compare in a meaningful way. (BTW: I also tried NetBSD 2.0.2, but its ldconfig is to different.)
File Added:
msg51499 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2006-12-20 18:43
Unfortunately I'm unable to review or work on this patch *this year*.  I will definitely take a look in January.  Sorry.
msg51500 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2007-01-09 20:01
mkam, I was eventually able to test out your patch.
I have virtual machines running Freebsd6.0, NetBSD3.0, and OpenBSD3.9.
The output from "print find_library('c'), find_library('m')" on these systems is as follows:


If you think this is what is expected, I'm happy to apply the patch.  Or is there further work needed on it?  (Do you still need the output of "ldconfig -r" or whatever?)
msg51501 - (view) Author: Martin Kammerhofer (mkam) Date: 2007-01-10 11:58
The output looks good. The patch selects the numerically highest library version.
NetBSD is not handled by the patch but works through _findLib_gcc (which will also
be tried as a fallback strategy for Free/Open-BSD when ldconfig output parsing fails.)

I think the patch is ready for commit.
msg51502 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2007-01-12 20:11
Neal, I think this can go into the release25-maint branch since it repairs the ctypes.util.find_library function on BSD systems.  What do you think?
msg51503 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2007-01-12 20:21
Committed into trunk as revision 53402.  Thanks for the patch and the work on it.
msg51504 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2007-01-17 07:09
Thomas, I don't see any (public) API changes and this fixes a bug.  I don't see a reason not to fix this in 2.5.1.  If you are comfortable with fixing, apply the patch.  Also, please update Misc/NEWS.  Thanks!
msg51505 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2007-01-17 19:59
Thanks, Neal, and Martin, again.

Committed as r53471 (and r53473 for Misc/NEWS) in the release25-maint branch.
Date User Action Args
2006-12-07 13:29:00mkamcreate