Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(156225)

Side by Side Diff: Lib/ctypes/_aixutil.py

Issue 26439: ctypes.util.find_library fails when ldconfig/glibc not available (e.g., AIX)
Patch Set: Created 3 years, 5 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « no previous file | Lib/ctypes/__init__.py » ('j') | Lib/ctypes/__init__.py » ('J')
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 # Lib/ctype support for cdll interface to dlopen() for AIX
2 # Author: M Felt, aixtools.net, May 2016
3 #
4 # dlopen() is an interface to AIX initAndLoad() - these are documented at:
5 # https://www.ibm.com/support/knowledgecenter/ssw_aix_53/com.ibm.aix.basetechref /doc/basetrf1/dlopen.htm?lang=en
6 # https://www.ibm.com/support/knowledgecenter/ssw_aix_53/com.ibm.aix.basetechref /doc/basetrf1/load.htm?lang=en
7 #
8 # Some basics
9 # when given an argument such as -lFOO to the AIX loader and a symbol must be re solved
10 # the AIX ld searches LIBPATH for an archive named libFOO.a. When found, it then searches archive members
11 # for a member that resolves the symbol.
12 # When an archive (.a) is not found, the search starts again, but looking for a file named libFOO.so
13 #
14 # when a full or partial path information is given dlopen() does not search LIBP ATH - only the path specified
15 #
16 # ctypes._aixutil.find_library() searches archives for members
17 # if a member is not found it looks for "name.so" in the same directory paths
18 #
19 # find_library("libFOO.so") is not an intended usage (looks for libFOO.so.a(libF OO.so.so) or libFOO.so.so)
20 # That may be suitable for CDLL(), but not for find_library()
21 # Much code is written as: CDLL(find_library("libFOO"))
Martin Panter 2016/06/20 08:43:57 Can you point to an example? This seems like an AI
Michael.Felt 2016/08/23 16:58:29 I wrote these comments "early", and misread code.
22 # or : CDLL((find_library("FOO"))
23 #
24 # examples:
25 # aix.find_library("libiconv") => libiconv.a(shr4_64.o) || libiconv.a(shr4. o) (64 or 32 bit modes)
26 # aix.find_library("libintl") => libintl.a(libintl.so.1) or libintl.a(libi ntl.so.8) depending on the
27 # version of GNU gettext installed
28
29 # when packaging python - a packager (for AIX) may want to include "export LDFLA GS=-L${prefix}/lib"
30 # so that dlopen() will look in ${prefix}/lib as well /usr/lib, etc..
31
32 #
33 # NOTE: RTLD_NOW is AIX default, regardless (as of March 2016) (set above!)
34 # NOTE: RTLD_MEMBER is needed by dlopen to open a shared object from within an a rchive
35 #
36 # find_library() returns the archive(member) string needed by dlopen() to open a member whenever possible
37 # only returning a file - when available - if the archive search fails
38 # directory path is returned only when directory path information was supplied
39 #
40 # Note: in the functions below local parameters might be declared to ensure scop e is local
41
42 import re, os, sys
43
44 # executable "size" is important
45 # archives can contain both 32 and 64-bit sizes - with different names
46 # e.g., shr.o (32-bit), shr_64.o (64-bit)
47 # Note: the names may be equal, e.g., libssl.so
48 # shared objects may have different search paths (e.g., /usr/lib (32-bit), /usr/ lib64 (64-bit))
49 # especially when they have dependancies on files rather than archives
50 def aixABI():
51 if (sys.maxsize < 2**32):
52 return 32
53 else:
54 return 64
55
56 # return striped output of /usr/bin/dump ... -H ...
57 def get_dumpH(object, p=None, lines=None):
58 import subprocess
59 p = subprocess.Popen(["/usr/bin/dump", "-X%s"%aixABI(), "-H", object],
60 universal_newlines=True, stdout=subprocess.PIPE)
61 # not interested in blank lines, leading/trailing spaces
62 lines=[line.strip() for line in p.stdout.readlines() if line.strip()]
63 # close process and wait (verify) it has completed
64 p.stdout.close()
65 p.wait()
66 # print lines
67 return lines
68
69 # process dumpH output
70 # return list of members with loader sections, i.e., shareable
71 # An excerpt of (stripped) dump -X64 -H output looks like:
72 # Note: a third # is part of the dump output!
73
74 ##/usr/lib/libc.a[vsaveres_64.o]:
75 ##Loader section is not available
76 ##/usr/lib/libc.a[ptrgl_64_64.o]:
77 ##Loader section is not available
78 ##/usr/lib/libc.a[shr_64.o]:
79 ##***Loader Section***
80 ##Loader Header Information
81 ##VERSION# #SYMtableENT #RELOCent LENidSTR
82 ##0x00000001 0x00000ab4 0x00002c63 0x0000002d
83 ###IMPfilID OFFidSTR LENstrTBL OFFstrTBL
84 ##0x00000003 0x0003c748 0x00009121 0x0003c775
85 ##***Import File Strings***
86 ##INDEX PATH BASE MEMBER
87 ##0 /usr/lib:/lib
88 ##1 / unix
89 ##2 libcrypt.a shr_64.o
90 ##/usr/lib/libc.a[posix_aio_64.o]:
91 ##***Loader Section***
92 ##Loader Header Information
93
94 def get_shared(input, list=None, cpy=None, idx=0):
95 cpy=input
Martin Panter 2016/06/20 08:43:57 This seems redundant
96 list=[]
97 for line in input:
98 idx = idx + 1
99 # if line does not start with "/" - it is superfluous
100 if line.startswith("/"):
101 next = cpy.pop(idx)
Martin Panter 2016/06/20 08:43:57 This hurts my head. Perhaps rewrite it with a loop
102 # if next line does not contain the word "not", then it has loader i nformation
103 # and the member including leftmost "[" and rightmost "]" needs to b e put on the list
104 # e.g., [shr_64.o] from: "/usr/lib/libc.a[shr_64.o]:"
105 if not "not" in next:
Martin Panter 2016/06/20 08:43:57 "not" not in next
106 list.append(line[line.find("["):line.rfind(":")])
Martin Panter 2016/06/20 08:43:57 (r)index would be more robust than (r)find, which
107 return(list)
Martin Panter 2016/06/20 08:43:57 Don’t use brackets on return statements
108
109 # looking for ONE exact match, otherwise return None
110 def get_match(expr, lines, line=None, member=None):
111 matches = [m.group(0) for line in lines for m in [re.search(expr, line)] if m]
Martin Panter 2016/06/20 08:43:57 Please rewrite this without the confusing second “
112 if len(matches) == 1:
113 match=matches[0]
114 # extract member name between leftmost "[" and rightmost "]" and return
115 member = match[match.find("[")+1:match.find("]")]
Martin Panter 2016/06/20 08:43:57 Since you’re already using a regular expression, y
116 return member
117 else:
118 return None
119
120 # sometimes a generic name gets 64 appended to it, and AIX 64-bit
121 # archive members have different legacy names than 32-bit ones
122 # the leading and trailing "[" and "]", respectively, are part of the
123 # search string to distinquish between archive (or BASE) and MEMBER
124 # see "INDEX PATH BASE MEMBER" header informatin above for an explanation of pos itions
125 def get_member64(name, members):
126 # This routine is only called when a libFOO.so was not found earlier
127 # CHECKME: change logic to do that search here as well?
128 # Recall that member names are in square brackets []
129 # an old convention - insert _64 between libFOO and .so
130 # Note: assumption - lib has been prepended (if needed)
131 # and .so is not part of the name
132 # '\[%s_*64\.so\]' % name, -> has either _64 or 64 added to name
Martin Panter 2016/06/20 08:43:57 Question mark for optional single underscore: %s_?
133 # Note: "version" line succeeds here only when there is only one versioned m ember.
134 # '\[%s.so.[0-9]+.*\]' % name, -> needs versioning (when no version was spec ified)
Martin Panter 2016/06/20 08:43:57 This has new unescaped dots (.), in addition to th
135 # '\[%s_*64\.o\]' % name, -> legacy name AIX scheme for libFOO 64-bit . so
136 # '\[shr4*_*64.o\]']: -> legacy AIX names: shr_64.o, shr_64.o, shr6 4.o
Martin Panter 2016/06/20 08:43:57 Please explain if the first 4 (and underscore) are
137 for expr in [
138 '\[%s_*64\.so\]' % name,
Martin Panter 2016/06/20 08:43:57 Please go back to using raw strings: r'\[. . .\]'.
139 '\[%s.so.[0-9]+.*\]' % name,
140 '\[%s_*64\.o\]' % name,
141 '\[shr4*_*64.o\]']:
Martin Panter 2016/06/20 08:43:57 IMO it would be easier to read with the list of RE
142 member = get_match(expr, members)
143 if member:
144 return member
145 return None
146
147 # Get the most recent/highest numbered version - if it exists
148 # Only called if a versioned is requested - CHECK - superfluous due to find_libr ary() convention?
149 def get_version(name, members):
150 def _num_version(libname):
151 # "libxyz.so.CURRENT.REVISION.AGE" => [ CURRENT, REVISION, AGE ]
152 parts = libname.split(".")
153 nums = []
154 try:
155 while parts:
156 nums.insert(0, int(parts.pop()))
157 except ValueError:
158 pass
159 return nums or [ sys.maxsize ]
160
161 expr = '%s[_64]*.so.[0-9]+[0-9\.]*' % name
Martin Panter 2016/06/20 08:43:57 This looks suspicious. At a minimum it needs clari
162 versions = [m.group(0) for line in members for m in [re.search(expr, line)] if m]
163 if versions:
164 versions.sort(key=_num_version)
165 return versions[-1]
166 return None
167
168 # return an archive member matching name
169 # (versioned) .so members have priority over legacy AIX member name
170 #
171 # member names in in the dumpH output are in square brackets
172 # These get stripped by get_match() and regular parenthesis are added by the cal ler
173 def get_member(name, members, member=None):
174 # look first for a generic match
175 # '\[%s\]' % name, -> CONCEIVEABLE: specific member requested
176 # '\[%s\.so\]' % name -> most common, append .so to name
177 for expr in [
178 '\[%s\.so\]' % name]:
Martin Panter 2016/06/20 08:43:57 Redundant “for” loop with only one iteration
179 member = get_match(expr, members)
180 if member:
181 return member
182 if not member:
183 member = get_version(name, members)
184 if member:
185 return member
186 elif aixABI() == 64:
187 return(get_member64(name, members))
188 # 32-bit legacy names - shr4.o and shr.o
189 else:
190 for expr in [
191 '\[shr4.o\]',
192 '\[shr.o\]']:
193 member = get_match(expr, members)
194 if member:
195 return member
196 return None
Martin Panter 2016/06/20 08:43:57 Would make more sense indented under “else:”
197
198 def getExecLibPath_aix(libpaths=None, x=None, lines=None, line=None, skipping=No ne):
199 # if LIBPATH is defined, add it to the paths to search
200 # FIXME: maybe LD_LIBRARY_PATH should be checked as well
201 libpaths = os.environ.get("LIBPATH")
202 lines = get_dumpH(sys.executable)
203 skipping = True;
Martin Panter 2016/06/20 08:43:57 semicolons like this are not needed (there is anot
204 for line in lines:
205 if skipping == False:
206 x = line.split()[1]
207 if (x.startswith('/') or x.startswith('./') or x.startswith('../')):
208 if libpaths is not None:
209 libpaths = "%s:%s" % (libpaths, x)
210 else:
211 libpaths = x
212 elif line.find("INDEX PATH") == 0:
213 skipping = False;
214 return libpaths
215
216 # Considerations:
217 # A) is there PATH information (if so, check only that directory), else determin e paths
218 # B) if name ends with .so.digit(s) - a specific version is requested
219 ## if so, look for "generic" archive name + specific member name
220 # option C - common requests
221 # C.1) FOO -> libFOO.a(libFOO.so)
222 # C.2) FOO -> libFOO.a(libFOO.so.X.Y.Z - latest version)
223 # C.3) FOO -> libFOO.a((AIX legacy name))
224
225 ##### A) if an argument resolves as a path.filename, use as path part as "paths"
226 def find_parts(name):
227 pathe = name.rfind("/")
228 if pathe > 0:
229 parts = (name[0:pathe], name[pathe+1:])
230 paths = parts[0]
Martin Panter 2016/06/20 08:43:57 Just write this as paths = name[0:pathe] etc
231 name = parts[1]
232 else:
233 paths = getExecLibPath_aix()
234 if name.find("lib") != 0:
Martin Panter 2016/06/20 08:43:57 if not name.startswith("lib")
235 name = "lib%s" % name
236 return (paths, name)
237
238 def find_library(name, _name=None, lines=None, path=None, member=None, shlib=Non e):
239 parts = find_parts(name)
240 paths = parts[0]
241 _name = parts[1]
Martin Panter 2016/06/20 08:43:57 Please find more distinctive names for the “name”
242
243 ##### B) the default AIX behavior is to find a member within a .a "base" arc hive
244 ## FIXME: if name endswith(".so.\D+" then:
245 ## we are looking for a specific version, e.g., libssl.so.0.9.8 => libssl.a (libssl.so.0.9.8)
246 _base = _name.rsplit(".so")[0]
247 for dir in paths.split(":"):
248 # /lib is a symbolic link to /usr/lib, skip it
249 if dir == "/lib":
250 continue
251 basefile = os.path.join(dir, "%s.a" % _base)
252 if os.path.exists(basefile):
253 members = get_shared(get_dumpH(basefile))
254 member = get_member(_name, members)
255 if member != None:
256 if name.rfind("/") > 0:
Martin Panter 2016/06/20 08:43:57 I presume you are just trying to check for the sam
257 # PATH is significant
258 shlib = "%s/%s.a(%s)" % (parts[0], _base, member)
Martin Panter 2016/06/20 08:43:57 I’m confused, is parts[0] a list of search paths,
259 else:
260 # SEARCHing is significant
261 shlib = "%s.a(%s)" % (_base, member)
262 return shlib
263 ##### C) look for a FILE based on the request
264 # Assumptions:
265 # Since it is not in an archive - considered a special case
266 # when .so is in filename
267 # if .so is not in name, append .so to filename before testing
268 # If path information is included in "name" return the path,
269 # or the path.so if either exists
270 if name.rfind("/") > 0:
271 if os.path.exists(name):
272 return(name)
273 else:
274 # add .so to name, just in case
275 shlib = "%s.so" % name
276 if os.path.exists(name):
277 return(shlib)
278 # did not find it based on literal "name", so try extra possibilities
279 # based on _name
280 # if .so is specified to find_library() then something specific is wanted
281 # use the name as supplied
282 if _name.find(".so") > 0:
Martin Panter 2016/06/20 08:43:57 exact = _name.find(".so") > 0
283 _name = name.rsplit("/")[-1]
284 exact = 1
285 else:
286 exact = 0
287 for dir in paths.split(":"):
288 # /lib is a symbolic link to /usr/lib, skip it
289 if dir == "/lib":
290 continue
291 if exact == 1:
Martin Panter 2016/06/20 08:43:57 if exact:
292 shlib = os.path.join(dir, "%s" % _name)
293 else:
294 shlib = os.path.join(dir, "%s.so" % _base)
295 if os.path.exists(shlib):
296 if name.rfind("/") > 0:
297 # PATH is significant
298 return shlib
299 else:
300 # SEARCHing is significant
301 return shlib.rsplit("/")[-1]
302 # if we are here, we have not found anything plausible
303 return None
OLDNEW
« no previous file with comments | « no previous file | Lib/ctypes/__init__.py » ('j') | Lib/ctypes/__init__.py » ('J')

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+