classification
Title: ctypes find_library should search LD_LIBRARY_PATH on linux
Type: enhancement Stage: patch review
Components: ctypes Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: theller Nosy List: Brian.Larsen, amaury.forgeotdarc, belopolsky, jniehof, lukasz.langa, vinay.sajip, yaroslavvb
Priority: normal Keywords: patch

Created on 2010-09-30 14:45 by jniehof, last changed 2012-07-19 07:48 by vinay.sajip.

Files
File name Uploaded Description Edit
ctypes_ld_lib_path.patch jniehof, 2010-09-30 14:45 Patch: ctypes.find_library() search LD_LIBRARY_PATH on linux
new-patch-using-ld-and-test.diff vinay.sajip, 2012-06-04 21:12 Patch using ld rather than gcc and with a test review
Repositories containing patches
http://hg.python.org/sandbox/vsajip#fix9998
Messages (10)
msg117738 - (view) Author: Jonathan Niehof (jniehof) Date: 2010-09-30 14:45
It's come up on occasion (#2936, http://osdir.com/ml/python.ctypes/2008-05/msg00046.html) that ctypes find_library doesn't search LD_LIBRARY_PATH for libraries, which is different behaviour from the runtime linker. The attached patch adds this search.

Unfortunately I can't conceive of a reasonable unit test for this (compiling a shared library in setUp seems a bit overkill.)
msg138738 - (view) Author: Brian Larsen (Brian.Larsen) Date: 2011-06-20 16:28
Hey I have this problem too.  I would love to see this fixed.
msg141079 - (view) Author: Yaroslav Bulatov (yaroslavvb) Date: 2011-07-25 09:46
This causes problem for Freetype Python bindings on Linux
msg141093 - (view) Author: Jonathan Niehof (jniehof) Date: 2011-07-25 14:17
Yaroslav: does the patch cause problems, or the original issue? If the former, could you be specific so I can try and fix it?
msg141115 - (view) Author: Yaroslav Bulatov (yaroslavvb) Date: 2011-07-25 19:01
Sorry for confusion, I meant the original problem causes problems. I haven't tested the patch because my target machines don't have gcc
msg143255 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-08-31 14:38
There is a problem in this area of the code, not especially with the patch but with how gcc works (or doesn't). To illustrate:

-----------------------
Version info
-----------------------
vinay@eta-hardy:/tmp$ uname -a
Linux eta-hardy 2.6.24-29-generic #1 SMP Wed Aug 10 16:34:32 UTC 2011 i686 GNU/Linux
vinay@eta-hardy:/tmp$ gcc --version
gcc (GCC) 4.2.4 (Ubuntu 4.2.4-1ubuntu4)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-----------------------
Create an empty file:
-----------------------
vinay@eta-hardy:/tmp$ touch dummy.c
-----------------------
Compile a shared library from it:
-----------------------
vinay@eta-hardy:/tmp$ gcc -shared -o libdummy.so dummy.c
-----------------------
Invoke gcc with patched command line:
-----------------------
vinay@eta-hardy:/tmp$ gcc -L /tmp -Wl,t -o /dev/null -ldummy
/usr/bin/ld: t: No such file: No such file or directory
collect2: ld returned 1 exit status
-----------------------
It's not patch related, here's what happens with the unpatched line
-----------------------
vinay@eta-hardy:/tmp$ gcc -Wl,t -o /dev/null -ldummy
/usr/bin/ld: t: No such file: No such file or directory
collect2: ld returned 1 exit status
-----------------------
Calling ld directly works as expected. With no search path:
-----------------------
vinay@eta-hardy:/tmp$ ld -t -o /dev/null -ldummy
ld: mode elf_i386
ld: cannot find -ldummy
-----------------------
With the search path supplied:
-----------------------
vinay@eta-hardy:/tmp$ ld -t -L /tmp -o /dev/null -ldummy
ld: mode elf_i386
-ldummy (/tmp/libdummy.so)
ld: warning: cannot find entry symbol _start; not setting start address
vinay@eta-hardy:/tmp$ 

So, ISTM that the find_library code needs changing to use ld, else it will not work on all platforms.
msg162296 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-06-04 21:11
I added an updated patch which uses ld, and added a test (Linux/Unix only).
msg165794 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2012-07-18 19:53
What changed since http://bugs.python.org/issue2936?
msg165806 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2012-07-18 21:36
> What changed since http://bugs.python.org/issue2936?

Well for one thing, #2936 was a bug report ("behavior") but this is an enhancement request. It seems that if you don't know exactly what a library is called, you can't use find_library to locate it even if it happens to be on LD_LIBRARY_PATH - and if you don't have the name, you can't load it using cdll.LoadLibrary.

ISTM LoadLibrary wants the full library name (including extension, on Linux); find_library helps determine that in a cross-platform way (since you can just give it the stem of the library rather than the whole filename).
msg165807 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2012-07-18 21:40
Fair enough. From what I've gathered though, it looks like the window for inclusion in 3.3 is already closed, right?
History
Date User Action Args
2012-07-19 07:48:22vinay.sajipsetversions: + Python 3.4, - Python 3.3
2012-07-18 21:40:20lukasz.langasetmessages: + msg165807
2012-07-18 21:36:49vinay.sajipsetmessages: + msg165806
2012-07-18 19:53:56lukasz.langasetnosy: + lukasz.langa
messages: + msg165794
2012-06-04 21:12:15vinay.sajipsetfiles: + new-patch-using-ld-and-test.diff
2012-06-04 21:11:21vinay.sajipsethgrepos: + hgrepo130
messages: + msg162296
stage: patch review
2011-08-31 14:38:25vinay.sajipsetnosy: + vinay.sajip
messages: + msg143255
2011-07-25 19:01:26yaroslavvbsetmessages: + msg141115
2011-07-25 14:17:41jniehofsetmessages: + msg141093
2011-07-25 09:46:36yaroslavvbsetmessages: + msg141079
2011-07-25 09:44:42yaroslavvbsetnosy: + yaroslavvb
2011-06-20 16:50:14r.david.murraysetnosy: + amaury.forgeotdarc, belopolsky, - theller

title: find_library should search LD_LIBRARY_PATH on linux -> ctypes find_library should search LD_LIBRARY_PATH on linux
2011-06-20 16:28:47Brian.Larsensetnosy: + Brian.Larsen
messages: + msg138738
2011-03-09 02:59:36terry.reedysetnosy: theller, jniehof
versions: + Python 3.3, - Python 3.1, Python 2.7, Python 3.2
2010-09-30 14:45:07jniehofcreate