CDLL does not use same paths as util.find_library #62702

shjohnsonpi · 2013-07-18T22:11:23Z

BPO	18502
Nosy	@meadori, @vadmium

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-07-18.22:11:22.612>
labels = ['ctypes', 'type-bug']
title = 'CDLL does not use same paths as util.find_library'
updated_at = <Date 2016-04-25.01:53:50.735>
user = 'https://bugs.python.org/shjohnsonpi'

bugs.python.org fields:

activity = <Date 2016-04-25.01:53:50.735>
actor = 'martin.panter'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2013-07-18.22:11:22.612>
creator = 'shjohnson.pi'
dependencies = []
files = []
hgrepos = []
issue_num = 18502
keywords = []
message_count = 4.0
messages = ['193332', '193577', '193584', '264143']
nosy_count = 3.0
nosy_names = ['meador.inge', 'martin.panter', 'shjohnson.pi']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue18502'
versions = ['Python 3.4']

shjohnsonpi · 2013-07-18T22:11:22Z

CDLL does not use the same paths as find_library and thus you can *find* a library, but you can't necessarily use it.

In my case, I had SDL2 in /usr/local/lib. find_library correctly gets
the name, but does not return the path. CDLL apparently does not search
/usr/local/lib and fails.

Python 3.4.0a0 (default:5a6cdc0d7de1, Jul 18 2013, 17:55:27) 
[GCC 4.7.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library("SDL2")
'libSDL2-2.0.so.0'
>>> CDLL(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/steven/Programming/cpython/Lib/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libSDL2-2.0.so.0: cannot open shared object file: No such file or directory
>>> CDLL("/usr/local/lib/libSDL2.so")
<CDLL '/usr/local/lib/libSDL2.so', handle 22f89c0 at 7fc966da5ae8>

meadori · 2013-07-23T01:32:10Z

On Linux gcc and ld are used to implement 'find_library' and 'dlopen' is used to implement 'CDLL'. ld searches /usr/local/lib. 'dlopen' might not if the LD_LIBRARY_PATH isn't set up to do so. For example:

[meadori@li589-207 cpython]$ ls /usr/local/lib
libfoo.so  libfoo.so.1  libfoo.so.1.0  linode
[meadori@li589-207 cpython]$ ./python 
Python 3.4.0a0 (default:d6213012d87b, Jul 23 2013, 01:00:24) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library('foo')
'libfoo.so.1'
>>> CDLL(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/meadori/src/cpython/Lib/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)OSError: libfoo.so.1: cannot open shared object file: No such file or directory
>>> quit()
[meadori@li589-207 cpython]$ echo $LD_LIBRARY_PATH

[meadori@li589-207 cpython]$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
[meadori@li589-207 cpython]$ ./python 
Python 3.4.0a0 (default:d6213012d87b, Jul 23 2013, 01:00:24) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> import ctypes.util
>>> ctypes.util.find_library('foo')
'libfoo.so.1'
>>> CDLL(_)
<CDLL 'libfoo.so.1', handle 28e4b50 at 7f31e062b608>

I am OK with this behavior in ctypes since it matches the system behavior with 'gcc' and 'dlopen'. So, I would fixup LD_LIBRARY_PATH or /etc/ld.so.conf if you want /usr/local/lib to be searched at load time.

shjohnsonpi · 2013-07-23T04:25:25Z

Should the docs then be changed for find_library to inform that "It is not guaranteed that a found library can be opened" (at least for linux)?

Or would a feature request for full paths from find_library for linux be more appropriate?

It seems hackish to me, if a program can find a library, but can't use it without asking the user to change some environment variables or testing /usr/local/lib.

vadmium · 2016-04-25T01:53:50Z

Documenting this quirk seems reasonable to me if you want to suggest a patch (and we don’t agree to fix it).

See bpo-21042 for returning full paths on Linux. We tried to make this change, but it wasn’t easy to do it consistently for arbitrary ABIs, so I backed it out.

See bpo-9998 discussing changing find_library() to emulate a run-time linker, rather than a compile-time linker.

shjohnsonpi mannequin added topic-ctypes type-bug An unexpected behavior, bug, or error labels Jul 18, 2013

ezio-melotti transferred this issue from another repository Apr 10, 2022

cboulay mentioned this issue Feb 13, 2023

util.find_library and CDLL do not use the same path --> library found but not loaded labstreaminglayer/pylsl#48

Open

cboulay mentioned this issue Aug 1, 2023

Linking issues on Ubuntu 20 for cthulhu facebookresearch/labgraph#2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CDLL does not use same paths as util.find_library #62702

CDLL does not use same paths as util.find_library #62702

shjohnsonpi mannequin commented Jul 18, 2013

shjohnsonpi mannequin commented Jul 18, 2013

meadori commented Jul 23, 2013

shjohnsonpi mannequin commented Jul 23, 2013

vadmium commented Apr 25, 2016

CDLL does not use same paths as util.find_library #62702

CDLL does not use same paths as util.find_library #62702

Comments

shjohnsonpi mannequin commented Jul 18, 2013

shjohnsonpi mannequin commented Jul 18, 2013

meadori commented Jul 23, 2013

shjohnsonpi mannequin commented Jul 23, 2013

vadmium commented Apr 25, 2016