Author Michael.Felt
Recipients Brian.Larsen, Daniel.Blanchard, Michael.Felt, Pau Tallada, amaury.forgeotdarc, belopolsky, jniehof, lukasz.langa, martin.panter, vinay.sajip, yaroslavvb
Date 2016-05-10.22:02:52
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1462917772.79.0.524216535528.issue9998@psf.upfronthosting.co.za>
In-reply-to
Content
In https://bugs.python.org/issue26439 I have been working on this for AIX - as the default behavior was to depend on two things:
a) ldconfig -p (which generally does not exist on AIX, and I doubt it will know about the non-gnu libraries
b) that the objects are files rather than members of an archive

For python cdll() to work (as find_library generally did not) - the work-around was to extract archive members into standard directories and/or hard-code unique AIX names. But again, always as files, as CDLL did not modify the dlopen() mode to permit loading a shared library from an archive.

re: this issue and behavioral differences between find_library and cdll() (rather dlopen()) searches - a linker is different from a dlopen. When compiling an argument such as -lc -- on AIX -- tells the linker to look in libc.a for any member (of the right size) that satisfies an unknown symbol. That archive (base) and member (member) name is remembered in the "loader section" during linking. Note that static members have no loader section of their own, nor do they add one to the object (program or shared library) being linked.
As a compatibility feature - when a member is not found in an archive the AIX rtl (run-time-linker) also looks for a file named libFOO.so (e.g., libc.so) - this grace behavior is why CDLL() has worked somewhat using default behavior in Lib/ctypes.

As to LD_LIBRARY_PATH and LIBPATH. In AIX terms, these are environment variables to supplement the library search path - only when no PATH information is provided (i.e., cdll("./libc.so") will only look in the current directory of the process, and only for the file libc.so)
The default search path is a list of the directories used to link the executable (e.g., python). For me that list is:                         ***Import File Strings***
INDEX  PATH                          BASE                MEMBER
0      /usr/vac/lib:/usr/lib:/lib

In other words, python, by default, is only going to look in very standard locations - even though it knows it's prefix is not /usr.

There are two solutions - and I will make changes as appropriate.
a) automate setting an environment variable if it is not already set to something like 
 _os.environ['LIBPATH'] = "%s/lib" % _sys.prefix
and not something "arbitrary" such as /usr/local/lib
b) modify the LDFLAGS when packaging to include
-L${prefix}/lib which will have the effect of changing the default search path via -W,-blibpath=/usr/vac/lib:/usr/lib:/lib:${prefix}/lib


In any case, in my patch for AIX in ctypes the two will work identical (cdll() even calls aix.find_library() to simplify portability between GNU/posix environments that expect or support only filenames to work with AIX archive packaging.

My apologies for the wall of text. This is not a simple subject.
History
Date User Action Args
2016-05-10 22:02:53Michael.Feltsetrecipients: + Michael.Felt, vinay.sajip, amaury.forgeotdarc, belopolsky, lukasz.langa, jniehof, Brian.Larsen, yaroslavvb, martin.panter, Daniel.Blanchard, Pau Tallada
2016-05-10 22:02:52Michael.Feltsetmessageid: <1462917772.79.0.524216535528.issue9998@psf.upfronthosting.co.za>
2016-05-10 22:02:52Michael.Feltlinkissue9998 messages
2016-05-10 22:02:52Michael.Feltcreate