Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(624)

#9998: ctypes find_library should search LD_LIBRARY_PATH on linux

Can't Edit
Can't Publish+Mail
Start Review
Created:
5 years, 6 months ago by jniehof
Modified:
1 year, 4 months ago
Reviewers:
vadmium+py, vinay_sajip, aixtools
CC:
Vinay Sajip, amaury.forgeotdarc, sasha, meadori, lukasz.langa, jniehof_lanl.gov, devnull_psf.upfronthosting.co.za, balarsen_gmail.com, yaroslavvb_gmail.com, Martin Panter, Michael.Felt, pautallada_gmail.com
Visibility:
Public.

Patch Set 1 #

Patch Set 2 #

Total comments: 42

Patch Set 3 #

Total comments: 2

Patch Set 4 #

Patch Set 5 #

Unified diffs Side-by-side diffs Delta from patch set Stats Patch
Doc/library/ctypes.rst View 1 2 3 4 2 chunks +12 lines, -5 lines 0 comments Download
Lib/ctypes/test/test_find.py View 1 2 3 4 1 chunk +43 lines, -0 lines 0 comments Download
Lib/ctypes/util.py View 1 2 3 4 1 chunk +25 lines, -1 line 0 comments Download

Messages

Total messages: 6
Martin Panter
https://bugs.python.org/review/9998/diff/18007/Doc/library/ctypes.rst File Doc/library/ctypes.rst (right): https://bugs.python.org/review/9998/diff/18007/Doc/library/ctypes.rst#newcode1242 Doc/library/ctypes.rst:1242: similar to what the compiler/linker does (on platforms with ...
1 year, 4 months ago #1
Vinay Sajip
Thanks for the review. I've addressed most of the comments, and will upload a new ...
1 year, 4 months ago #2
Martin Panter
https://bugs.python.org/review/9998/diff/18007/Lib/ctypes/test/test_find.py File Lib/ctypes/test/test_find.py (right): https://bugs.python.org/review/9998/diff/18007/Lib/ctypes/test/test_find.py#newcode98 Lib/ctypes/test/test_find.py:98: self.assertEqual(p.returncode, 0) On 2016/07/29 20:38:17, Vinay Sajip wrote: > ...
1 year, 4 months ago #3
aixtools_gmail.com
Hope this publishes all my comments. Still getting used to the interface. In short - ...
1 year, 4 months ago #4
Vinay Sajip
Thanks for the additional comments. New patchset incoming. http://bugs.python.org/review/9998/diff/18007/Doc/library/ctypes.rst File Doc/library/ctypes.rst (right): http://bugs.python.org/review/9998/diff/18007/Doc/library/ctypes.rst#newcode1242 Doc/library/ctypes.rst:1242: similar ...
1 year, 4 months ago #5
Martin Panter
1 year, 4 months ago #6
https://bugs.python.org/review/9998/diff/18007/Lib/ctypes/util.py
File Lib/ctypes/util.py (right):

https://bugs.python.org/review/9998/diff/18007/Lib/ctypes/util.py#newcode297
Lib/ctypes/util.py:297: p = subprocess.Popen(cmd, stdout=subprocess.PIPE,
On 2016/08/07 18:15:10, Vinay Sajip wrote:
> On 2016/07/30 02:31:44, vadmium wrote:
> > The “ld” command. If “gcc” is missing, the current code will return None,
and
> > pass it through so that find_library() returns None. If “gcc” is not
> installed,
> > it is likely that “ld” is also missing, but with your code it looks like
> > find_library() will now raise FileNotFoundError.
> 
> Isn't "ld" a basic part of the operating system on POSIX? I assumed it was,
but
> I can certainly catch the exception and return None in that case.

I only really have experience with Linux, where it is usually part of a compiler
package. On Arch Linux:

$ pacman --query --owns ld gcc
/sbin/ld is owned by binutils 2.25-5
/sbin/gcc is owned by gcc 5.1.0-5

I’m pretty sure it is possible to set up Linux and install Python without either
package (although if you have GCC installed, you will also get binutils as a
dependency). Not sure what the status with Posix is, but it is not listed at
<http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html>, so it may
not be a requirement at all.

https://bugs.python.org/review/9998/diff/18007/Lib/ctypes/util.py#newcode302
Lib/ctypes/util.py:302: res = re.search(expr, out.decode(encoding))
On 2016/08/07 18:15:10, Vinay Sajip wrote:
> On 2016/07/30 02:31:44, vadmium wrote:
> > Surrogateescape is the usual choice. “Replace” should be fine for most
> > instances, e.g. where the undecodable bytes are in other bits of the output.
> But
> > fsdecode() would use surrogateescape, which I suspect would even work in the
> > unlikely case that the library filename contained an undecodable byte (e.g.
it
> > could have been passed to Python via a command-line argument).
> 
> We can never be sure (in general) that we will handle all decoding errors
> optimally, since we can't always be sure if the binary output was encoded in a
> systematic way.

That is why I think os.fsdecode() is the most optimal and consistent way to
decode. In my experiments with Linux and GCC, if you override the locale with
ASCII (LC_ALL=C), find_library() still works for non-ASCII file names if
surrogateescape is used.

Using Lib/ctypes/util.py with the unreleased improvements from Issue 22636, and
normal UTF-8 locale. Make a library with non-ASCII file name (SONAME is
unchanged):
$ sudo cp /lib/libreadline.so /lib/libblauä.so

Find_library() works when using UTF-8:
$ python3 -c 'import util, sys; print(util.find_library(sys.argv[1]))' blauä
libreadline.so.6

With the recent changes, find_library() now also works using surrogateescape:
$ LC_ALL=C python3 -c 'import util, sys; print(util.find_library(sys.argv[1]))'
blauä
libreadline.so.6

> I have no problem using 'surrogateescape' instead of 'replace', but if
> universal_newlines is true, doesn't this become a moot point?

Yes, I think you are stuck with "strict" with universal_newlines=True.
Sign in to reply to this message.

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7