classification
Title: ctypes.util incorrectly fails for libraries without DT_SONAME
Type: behavior Stage: patch review
Components: ctypes Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeremy.Huntwork, Kylie McClain, Richard Eames, jcastillo2nd, martin.panter, tianon, twiggers, vstinner
Priority: normal Keywords: patch

Created on 2014-06-01 03:28 by Jeremy.Huntwork, last changed 2019-02-11 11:13 by vstinner.

Pull Requests
URL Status Linked Edit
PR 10453 closed python-dev, 2018-11-10 20:42
PR 10455 closed jcastillo2nd, 2018-11-10 21:52
PR 10460 open jcastillo2nd, 2018-11-11 10:23
PR 10461 closed jcastillo2nd, 2018-11-11 10:26
PR 10462 closed jcastillo2nd, 2018-11-11 10:27
Messages (7)
msg219481 - (view) Author: Jeremy Huntwork (Jeremy.Huntwork) Date: 2014-06-01 03:28
On my system, the C library (musl) intentionally does not include a SONAME entry.

This method in particular fails: http://hg.python.org/cpython/file/076705776bbe/Lib/ctypes/util.py#l133

The function seems to jump through some hoops which may not be necessary. Is there a reason for wanting particularly to use the SONAME entry for the lib?

In my system the following works as a replacement for _get_soname:

return os.path.basename(os.path.realpath(f))
msg264141 - (view) Author: Kylie McClain (Kylie McClain) Date: 2016-04-25 01:09
This is still a problem on musl distributions as of 2.7.11. Are there any plans to fix this?

Patch used by Alpine Linux:
http://git.alpinelinux.org/cgit/aports/tree/main/python/musl-find_library.patch
msg264148 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-04-25 03:20
On Linux, the find_library() function is documented to return “the filename of the library file”, but in reality it seems it return the soname, and therefore breaks if there is no soname. I do not know why we extract the soname, but it has been that way at least since ctypes was added to Python 2.5.

What do you intend to do with the result of find_library()? See also Issue 9998, especially about searching LD_LIBRARY_PATH. I am struggling to see robust use cases for find_library().
msg274361 - (view) Author: Thom Wiggers (twiggers) Date: 2016-09-04 13:47
This bug is still present in Python 3.5.

It breaks, probably among other things, this package: https://github.com/ahupp/python-magic/issues/114.
msg274384 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-09-05 02:03
I think it may be reasonable to change the code to return the library file name if no soname can be found.

In the long term, I think always returning the filename rather than soname might be reasonable. Or even returning the full path, which we tried in Issue 21042 (but rolled back because the solution so far is not consistent across platforms).
msg321772 - (view) Author: Tianon (tianon) Date: 2018-07-16 21:55
This was reported on the Docker image for Python in https://github.com/docker-library/python/issues/111, with the note that it affects the Twisted inotify implementation, so it'd be really neat to see a proper patch in Python (instead of the very musl-/Alpine-assuming patch found in https://github.com/alpinelinux/aports/blob/202f4bea916b0cf974b38ced96ab8fca0b192e3f/main/python2/musl-find_library.patch). <3
msg330668 - (view) Author: Javier Castillo II (jcastillo2nd) * Date: 2018-11-29 08:01
The PR 10460 ( for 3.8 ) patches the search attempts to leverage LD_LIBRARY_PATH for the Linux case and catches the case after SONAME, gcc and ldconfig behaviors fail.

While this may not be set in all environments ( like the musl based Alpine docker image ) setting the environment variable is within the user's control and when properly set does return found libraries.

PR 10461 and PR 10462 are closed for now, as backports to 2.7 and 3.7 need to have the 3.8 accepted first before being reviewed.
History
Date User Action Args
2019-02-11 11:13:58vstinnersetnosy: + vstinner
2018-11-29 08:01:57jcastillo2ndsetnosy: + jcastillo2nd
messages: + msg330668
2018-11-11 10:27:57jcastillo2ndsetpull_requests: + pull_request9736
2018-11-11 10:26:21jcastillo2ndsetpull_requests: + pull_request9735
2018-11-11 10:23:15jcastillo2ndsetpull_requests: + pull_request9734
2018-11-10 21:52:29jcastillo2ndsetpull_requests: + pull_request9730
2018-11-10 20:42:00python-devsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request9728
2018-07-16 21:55:32tianonsetnosy: + tianon
messages: + msg321772
2017-09-18 20:30:09Richard Eamessetnosy: + Richard Eames
2016-09-05 02:03:18martin.pantersetstage: needs patch
messages: + msg274384
versions: + Python 3.6
2016-09-04 13:47:42twiggerssetnosy: + twiggers

messages: + msg274361
versions: + Python 3.5
2016-04-25 03:20:15martin.pantersetnosy: + martin.panter
messages: + msg264148
2016-04-25 01:09:18Kylie McClainsetnosy: + Kylie McClain
messages: + msg264141
2014-06-01 03:28:22Jeremy.Huntworkcreate