classification
Title: CDLL does not use same paths as util.find_library
Type: behavior Stage:
Components: ctypes Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: martin.panter, meador.inge, shjohnson.pi
Priority: normal Keywords:

Created on 2013-07-18 22:11 by shjohnson.pi, last changed 2016-04-25 01:53 by martin.panter.

Messages (4)
msg193332 - (view) Author: Steven Johnson (shjohnson.pi) Date: 2013-07-18 22:11
CDLL does not use the same paths as find_library and thus you can *find* a library, but you can't necessarily use it.

In my case, I had SDL2 in /usr/local/lib. find_library correctly gets
the name, but does not return the path. CDLL apparently does not search
/usr/local/lib and fails.

Python 3.4.0a0 (default:5a6cdc0d7de1, Jul 18 2013, 17:55:27) 
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library("SDL2")
'libSDL2-2.0.so.0'
>>> CDLL(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/steven/Programming/cpython/Lib/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libSDL2-2.0.so.0: cannot open shared object file: No such file or directory
>>> CDLL("/usr/local/lib/libSDL2.so")
<CDLL '/usr/local/lib/libSDL2.so', handle 22f89c0 at 7fc966da5ae8>
msg193577 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2013-07-23 01:32
On Linux gcc and ld are used to implement 'find_library' and 'dlopen' is used to implement 'CDLL'.  ld searches /usr/local/lib.  'dlopen' might not if the LD_LIBRARY_PATH isn't set up to do so.  For example:


[meadori@li589-207 cpython]$ ls /usr/local/lib
libfoo.so  libfoo.so.1  libfoo.so.1.0  linode
[meadori@li589-207 cpython]$ ./python 
Python 3.4.0a0 (default:d6213012d87b, Jul 23 2013, 01:00:24) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library('foo')
'libfoo.so.1'
>>> CDLL(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/meadori/src/cpython/Lib/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)OSError: libfoo.so.1: cannot open shared object file: No such file or directory
>>> quit()
[meadori@li589-207 cpython]$ echo $LD_LIBRARY_PATH

[meadori@li589-207 cpython]$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
[meadori@li589-207 cpython]$ ./python 
Python 3.4.0a0 (default:d6213012d87b, Jul 23 2013, 01:00:24) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library('foo')
'libfoo.so.1'
>>> CDLL(_)
<CDLL 'libfoo.so.1', handle 28e4b50 at 7f31e062b608>


I am OK with this behavior in ctypes since it matches the system behavior with 'gcc' and 'dlopen'.  So, I would fixup LD_LIBRARY_PATH or /etc/ld.so.conf if you want /usr/local/lib to be searched at load time.
msg193584 - (view) Author: Steven Johnson (shjohnson.pi) Date: 2013-07-23 04:25
Should the docs then be changed for find_library to inform that "It is not guaranteed that a found library can be opened" (at least for linux)?

Or would a feature request for full paths from find_library for linux be more appropriate?

It seems hackish to me, if a program can find a library, but can't use it without asking the user to change some environment variables or testing /usr/local/lib.
msg264143 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-25 01:53
Documenting this quirk seems reasonable to me if you want to suggest a patch (and we don’t agree to fix it).

See Issue 21042 for returning full paths on Linux. We tried to make this change, but it wasn’t easy to do it consistently for arbitrary ABIs, so I backed it out.

See Issue 9998 discussing changing find_library() to emulate a run-time linker, rather than a compile-time linker.
History
Date User Action Args
2016-04-25 01:53:50martin.pantersetnosy: + martin.panter
messages: + msg264143
2013-07-23 04:25:25shjohnson.pisetmessages: + msg193584
2013-07-23 01:32:11meador.ingesetnosy: + meador.inge
messages: + msg193577
2013-07-18 22:11:22shjohnson.picreate