Author aixtools@gmail.com
Recipients David.Edelsohn, Michael.Felt, aixtools@gmail.com, martin.panter
Date 2016-06-08.21:39:16
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <bed1d3a0-19bc-d808-4936-7f14d984861e@gmail.com>
In-reply-to <1465050241.59.0.841731746859.issue26439@psf.upfronthosting.co.za>
Content
Answering this again - now that the new patch is ready.

On 04-Jun-16 16:24, Martin Panter wrote:
> Martin Panter added the comment:
>
> Okay here are some more thoughts about your latest patch:
>
> ## Automatic RTLD_MEMBER ##
>
> I was still uneasy about the automatic setting of RTLD_MEMBER. But I looked for how others handle this, and I found Libtool’s LTDL library, and Apache Portable Runtime (APR). Both have a similar (but stricter) automatic addition based on detecting the archive(member) notation:
>
> http://git.savannah.gnu.org/cgit/libtool.git/commit/libltdl/loaders/dlopen.c?id=8fa719e
> https://github.com/apache/apr/commit/c27f8d2
>
> So I guess I can accept this way, if you think it is better than my other ideas:
a) I think it is more in line with what I perceive as normal practice
  libFOO = CDLL(find_library("FOO"))
As CDLL() has always had a 'simple' string as input and not a "tuple".
I have also added some lines to test/test_loading.py to test direct 
calling of CDLL() with fixed strings and a test of os.maxsize
and in util.py - but using e.g., CDLL(find_library("c") as behavior is 
dependent on 32 or 64-bit mode

depending on mode - different output:
  note: find_library("libssl64") is expected to return None - as it 
would be "abnormal" to have an archive libssl64.a or a file libssl64.so

cd Lib/ctypes
../../python util.py
# 32-bit mode:
None
libc.a(shr.o)
libbz2.a(libbz2.so)
find_library("c")        returns: libc.a(shr.o)
find_library("libc")     returns: libc.a(shr.o)
find_library("libssl")   returns: libssl.a(libssl.so)
find_library("libssl64") returns: None
find_library("ssl")      returns: libssl.a(libssl.so)
find_library("libiconv") returns: libiconv.a(libiconv.so.2)
find_library("intl")     returns: libintl.a(libintl.so.8)
libcrypt.a(shr.o)
<CDLL 'libc.a(shr.o)', handle c at 300e8330>
<CDLL 'libcrypt.a(shr.o)', handle d at 300e8830>

# 64-bit mode:
None
libc.a(shr_64.o)
libbz2.a(libbz2.so.1)
find_library("c")        returns: libc.a(shr_64.o)
find_library("libc")     returns: libc.a(shr_64.o)
find_library("libssl")   returns: libssl.a(libssl64.so.1.0.0)
find_library("libssl64") returns: None
find_library("ssl")      returns: libssl.a(libssl64.so.1.0.0)
find_library("libiconv") returns: libiconv.a(shr4_64.o)
find_library("intl")     returns: libintl.a(libintl.so.1)
libcrypt.a(shr_64.o)
<CDLL 'libc.a(shr_64.o)', handle c at 7000000001269e8>
<CDLL 'libcrypt.a(shr_64.o)', handle d at 700000000126a20>

>
> # Always uses RTLD_MEMBER, never loads a plain file outside an archive
> name = "libcrypto.a(libcrypto.so.1.0.0)"
>
> # Other options, that could be returned by find_library() and would not conflict with a plain file name
> name = ("libcrypto.a", "libcrypto.so.1.0.0")  # (archive, member)
> name = ("libcrypto.a(libcrypto.so.1.0.0)", RTLD_MEMBER)  # (name, extra-flags)
>
> libcrypto = CDLL(name)
>
> ## find_library() modes ##
>
> In your find_library() function, you still have three parts. Can you confirm that each behaviour is intended:
I was being "Q&D" here, not changing the aixutils.py (now _aixutils.py). 
My intent is to be comparable with other "posix" behaviors.
So, if you believe I am still not compatible with their behavior, 
forgive me - but also shoot me!
>
> A) If I have a file called "crypto" in the current directory, find_library("crypto") returns "crypto". This does not seem right. On Linux, “gcc [. . .] -lcrypto” does not look for a file exactly called “crypto”.
>
> B) You are still stripping bits off the library name if it contains “lib” or a dot (.), so find_library("glib-2.0") is almost equivalent to find_library("b-2"). Isn’t this a bug?
>
> C) find_library("crypto") will return "/usr/lib/crypto" if such a file exists. Just like in A), this does not seem right to me.
All should be seen as bugs, and I hope I coded it correctly to not do 
this anymore.
> ## Other things ##
>
> * You don’t need to prefix most names with underscores, unless they could be confused with a public API. If you follow my earlier suggestion of renaming the new file to _aixutil.py (so it is obvious it is not a public module), then you can freely write “import re, os, sys”, etc.
I had missed, certainly not understood the context, before. aixutil.py 
is now _aixutil.py. Originally I had done this to make the diff in 
util.py much simplier, but also because I incorrectly thought CDLL() was 
frequently called with "foo" or "libfoo". In short, trying to prevent a 
non-existent problem.
__init__.py delta is also much much simpler to grasp.
> * No need to add the internal variable names to the function signatures. Just write find_library(name), and if you need to initialize a variable, do that in the body.
Oops - not removed those yet. That was done to be sure there was no 
global scope interference. If you feel it is vital they be removed - 
will be done.
>
> * I suggest to go over all the regular expressions, and either change them to plain string searching, or make sure special characters and external variables are escaped as necessary. A comment explaining what the RE is trying to do might help too.
Ugh. I actually hate string stuff. I will need to spend more time on 
that. FYI - after a lot of testing with various expr strings I found not 
using the 'raw' format worked better (read: I was probably making 
beginner errors, and made fewer when using "normal" back-slash escapes.

However, I shall add comments in the next pass (which I now conclude is 
unavoidable)

However #2 - I hope we are really really close to an acceptable patch.

Michael
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26439>
> _______________________________________
History
Date User Action Args
2016-06-08 21:39:17aixtools@gmail.comsetrecipients: + aixtools@gmail.com, martin.panter, David.Edelsohn, Michael.Felt
2016-06-08 21:39:17aixtools@gmail.comlinkissue26439 messages
2016-06-08 21:39:16aixtools@gmail.comcreate