New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AIX shared library extension modules installation broken #70011
Comments
AIX requires helper scripts to build Python shared extension modules. The definitions and Makefile installation rules have bitrotted. Makefile.pre.in:
except python.exp is created in Modules/python.exp, not Programs.exp
Post-substitution Makefile.pre: BINLIBDEST= configure.ac:
which is forever enshrined in _sysconfigdata.py In other words, configure sets LDSHARED to $(BINLIBDEST)/config, which does not exist in current installations. And Makefile installs the files in LIBPL, which is based on LIBDEST (prefix and exec_prefix could be different). And the files are not installed, because the installation tests Programs/python.exp instead of Modules/python.exp. Changing Makefile.pre.in to test Modules/python.exp is easy enough as a partial fix. What is the preferred location to install the files so that configure.ac and Makefile.pre.in can be adjusted to match each other? |
It looks like the Programs/python.exp change was done accidentally in revision 88a532a31eb3 (bpo-18093), so we should be safe reverting the relevant lines back to Modules/python.exp. The configure script currently sets LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}" However there is a contradictory comment in the makefile: # This goes into $(exec_prefix) It looks like the comment came first. This isn’t my area of expertise, but my understanding is |
# Install prefix for architecture-dependent files python.exp is not architecture dependent, although it only is useful on AIX target. It is essentially equivalent to a list of symbols with ELF global, non-hidden visibility. It is less confusing if the list is co-located with the scripts that use it. LIBPL is fine with me. configure.ac and Makefile.pre.in must match. |
FYI: when build and src are the same directory, there is no error message. However, when src and build are in seperate directories (e.g., build = '.', the following message occurs MANY times: unable to execute '../src/python-2.7.11/Modules/ld_so_aix': No such file or directory during configure the file root@x064:[/data/prj/aixtools/python-2.7.11]ls -l Modules/ld*aix* Just guessing, but a line such as: using $(srcdir) is not correct. Further, a find in $(srcdir) for ld_so_aix returns: So, $(srcdir) seems to be wrong, by definition. For now, I am copying |
hmm. needs patch. For what? One comment says a change will be undone. If a patch is expected for src/builddir issue: a) I sinned - that is a new issue, and should be posted seperately. rather than a patch i can on that is see if the same issue occurs with linux and post a new issue. So, my question would be - was a rollback done - if yes, perhaps a comment to verify and then close this issue. |
Let's start with this patch to revert the change mentioned in msg256136 to look for python.exp in Modules -- the test, the source for the file, and the location to delete the file. A follow-up patch will fix the data in _sysconfigdata.py. |
Patch1 looks fine to me, though I will have to rely on you people to verify that it does what it’s supposed to. Do you want me to commit it straight away, or wait for your follow-up patch? |
Yes, please apply Patch 1 that reverts the mistaken change of revision 88a532a31eb3 . I want to work through this incrementally so that it's clear to reviewers. |
New changeset 58c8cae6c61a by Martin Panter in branch '3.5': New changeset 4d4b5b978b7e by Martin Panter in branch 'default': |
The second of two patches. This patch changes the definition of LDSHARED for AIX in configure to reference the matching installed location as defined in Makefile.pre.in by Patch1. The definition from configure propagates into _sysconfigdata.py. This change will (further) break test_distutils, but the testcase is wrong on AIX. And, for AIX, a correct test result clearly does not correspond to correct behavior. The patch affects both configure.ac and configure because the corresponding change to configure is obvious. The person who installs the patch may regenerate configure, as appropriate. |
Independently, I have created a similar patch as well and can also verify that it does what it's suppose to do.
I can also confirm that Patch2 is necessary as well and does what it's suppose to do, having creating this patch independently as well. |
It looks like the change from plain config to config-<ABI> was made in bpo-9807 (r86731, 3.2). So the patch seems reasonable to me; it is just catching up with that change. This bug was marked for 2.7 as well. Is there anything that needs to be done for 2.7? How does patch 2 make the test_distutils situation worse? Is there anything that should be done to improve things first, or should that just be handled independently? I see that the 2.7 and 3.5 buildbots are currently failing with errors like distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit status 1 while 3.6 is failing with distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' failed with exit status 1 Presumably the 3.6 failure may be helped by Patch 2, since it is complaining about the plain “config” path. |
It would be great if both patches were applied to 2.7 also.
distutils.errors.LinkError: command './Modules/ld_so_aix' failed with exit status 1 while 3.6 is failing with distutils.errors.LinkError: command '/usr/local/lib/python3.6/config/ld_so_aix' failed with exit status 1 Presumably the 3.6 failure may be helped by Patch 2, since it is complaining about the plain “config” path. Patch2 makes test_distutils worse because LDSHARED refers to a file that only exists when Python is installed. If one tests in tree, the files don't exist. This may be related to the existence of BLDSHARED, used to build modules in the source tree, which can be overridden separately -- otherwise it defaults to LDSHARED. If the rest of the testsuite ever is cleaned up for AIX, one either could skip the parts of test_distutils that require the LDSHARED files or add special hooks in the test for AIX. As you point out, the change doesn't make the current testsuite results situation any worse. |
Patch 1 does not apply to 2.7. The offending commit 88a532a31eb3 was not made in that branch. Patch 2 may apply to 2.7, but I am not sure if it is needed or worthwhile. Again, the offending commit r86731 not made to 2.7. There is also a $(BINLIBDEST)/config reference for Be OS; surely if one is updated the other should too. David: I don’t understand “Patch2 makes test_distutils worse because LDSHARED refers to a file that only exists when Python is installed.” At the moment, doesn’t LDSHARED refer to a file that never exists? So at least the test will be improved when Python is installed, right? I looked into the test_distutils failures and here’s what I found:
|
Patch2 will not make test_distutils results worse, but the test results may not represent the true status of distutils on AIX if the matching Python version is not installed so that the test can find the files in the installed location. |
New changeset ca1ddd365f5f by Martin Panter in branch '3.5': New changeset 5a05c0eeefc3 by Martin Panter in branch 'default': |
Okay, the second patch is committed to 3.5+. Is everything working now (on 2.7, 3.5, 3.6), or is there more to do? |
I believe that everything is functioning correctly. |
On 10/08/2016 17:22, David Edelsohn wrote:
a) would appreciate your testig them with gcc - to be sure all is good Hope this helps (AIX :) ) Michael Links: |
On 18/09/2018 16:46, Michael Felt wrote:
Ignore this. If only I could remove stuff! Good day all. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: