classification
Title: _sysconfigdata.py wrong on AIX installations
Type: behavior Stage: resolved
Components: Build Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David.Edelsohn, eric.araujo, haubi, larry, pelson, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2013-06-17 00:33 by David.Edelsohn, last changed 2014-10-30 09:38 by pelson. This issue is now closed.

Files
File name Uploaded Description Edit
issue18235.patch David.Edelsohn, 2013-10-19 18:48 review
Messages (18)
msg191301 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-06-17 00:33
_sysconfigdata.py includes information about how to build extension modules. On AIX this requires a wrapper script to build shared libraries.  The file includes definitions like:

 'BLDSHARED': './Modules/ld_so_aix gcc -pthread -bI:./Modules/python.exp',
 'LDSHARED': './Modules/ld_so_aix gcc -pthread -bI:./Modules/python.exp',
 'LINKCC': './Modules/makexp_aix Modules/python.exp . libpython3.4dm.a;  gcc '
           '-pthread',
 'MAKESETUP': './Modules/makesetup',

which is correct in the build directory, but is not correct for the install directory.  The paths do not correspond to the installed location of ld_so_aix and makexp_aix in lib/pythonX.Y/config .
msg191385 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-06-18 01:26
This is the cause of the failures in test_distutils.
msg191598 - (view) Author: √Čric Araujo (eric.araujo) * (Python committer) Date: 2013-06-21 17:15
Can you suggest how to fix this?
msg191624 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-06-22 00:35
It looks like someone already tried to fix part of this, but reversed the assignment

diff -r a089a8b1f93d Lib/sysconfig.py
--- a/Lib/sysconfig.py  Fri Jun 21 18:37:02 2013 -0400
+++ b/Lib/sysconfig.py  Fri Jun 21 22:33:15 2013 -0700
@@ -368,7 +368,7 @@
     # -- these paths are relative to the Python source, but when installed
     # the scripts are in another directory.
     if _PYTHON_BUILD:
-        vars['LDSHARED'] = vars['BLDSHARED']
+        vars['BLDSHARED'] = vars['LDSHARED']
 
     # There's a chicken-and-egg situation on OS X with regards to the
     # _sysconfigdata module after the changes introduced by #15298:


BLDSHARED is relative to srcdir and LDSHARED is relative to the install directory BINLIBDEST.

LDSHARED=       $(BINLIBDEST)/config/ld_so_aix $(CC) -bI:$(BINLIBDEST)/config/python.exp $(PY_LDFLAGS)
BLDSHARED=      $(srcdir)/Modules/ld_so_aix $(CC) -bI:$(srcdir)/Modules/python.exp $(PY_LDFLAGS)
msg200476 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-10-19 18:48
Patch against current trunk
msg200479 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-10-19 19:02
With my patch applied, _sysconfig.py looks like

 'BLDSHARED': '/usr/local/lib/python3.4/config/ld_so_aix gcc -pthread '
              '-bI:/usr/local/lib/python3.4/config/python.exp',
 'LDSHARED': '/usr/local/lib/python3.4/config/ld_so_aix gcc -pthread '
             '-bI:/usr/local/lib/python3.4/config/python.exp',
msg200488 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-19 20:00
Ha, that code dates back to 2000 and 2001 (f1e40abbedb3 and c958678720fd, respectively).
msg200491 - (view) Author: Roundup Robot (python-dev) Date: 2013-10-19 20:07
New changeset c554194240fc by Antoine Pitrou in branch '3.3':
Issue #18235: Fix the sysconfig variables LDSHARED and BLDSHARED under AIX.
http://hg.python.org/cpython/rev/c554194240fc

New changeset e3ac917fcf9c by Antoine Pitrou in branch 'default':
Issue #18235: Fix the sysconfig variables LDSHARED and BLDSHARED under AIX.
http://hg.python.org/cpython/rev/e3ac917fcf9c
msg200492 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-19 20:08
Committed the patch. I trust you that you got it right!
msg200875 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-10-22 04:18
I think I see the source of the confusion. test_distutils fails because it runs in the build tree and without files installed. The test does not use the installed version of sysconfig, so it looks for ./Modules/ld_so_aix, which fails. The kludge from 2000 was trying to make the test work. But after installation, the kludge defines variables incorrectly to build Modules after Python is installed.

Because the distutils test is testing in tree and not the installed version of Python, it is not testing what the end user will experience. I am unsure about the best way to make the test pass.
msg202427 - (view) Author: Michael Haubenwallner (haubi) * Date: 2013-11-08 15:24
Actually, ld_so_aix is autogenerated from ld_so_aix.in into builddir, while makexp_aix is not.

Attached patch eventually might fix the test too?
msg202436 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-11-08 16:46
+1
msg205297 - (view) Author: Michael Haubenwallner (haubi) * Date: 2013-12-05 13:51
Kindly ping. Do you prefer another report for the aix-absbuilddir patch as this one is already closed?
msg205310 - (view) Author: David Edelsohn (David.Edelsohn) * Date: 2013-12-05 16:26
Do you want me to open a new issue or do you want to open a new issue?
msg205315 - (view) Author: Michael Haubenwallner (haubi) * Date: 2013-12-05 17:57
Erm, question target was the python committers. And I'd create the new issue if they prefer.
msg219670 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-03 08:42
issue#10656 is the out-of-source part already.
msg219888 - (view) Author: Michael Haubenwallner (haubi) * Date: 2014-06-06 15:23
Hmm... instead of reversing the order while keeping in _generate_posix_vars(), feels like it would have been better to move the code from 2000 back to _init_posix() where it originally was, without changing the order - because now for sysconfig within Python build, the working B-value of LDSHARED is lost.

Something like (note the function names):

--- a/Lib/sysconfig.py  Fri Jun 06 01:23:53 2014 -0500
+++ b/Lib/sysconfig.py  Fri Jun 06 17:11:02 2014 +0200
@@ -364,11 +364,6 @@ def _generate_posix_vars():
         if hasattr(e, "strerror"):
             msg = msg + " (%s)" % e.strerror
         raise OSError(msg)
-    # On AIX, there are wrong paths to the linker scripts in the Makefile
-    # -- these paths are relative to the Python source, but when installed
-    # the scripts are in another directory.
-    if _PYTHON_BUILD:
-        vars['BLDSHARED'] = vars['LDSHARED']
 
     # There's a chicken-and-egg situation on OS X with regards to the
     # _sysconfigdata module after the changes introduced by #15298:
@@ -409,6 +404,11 @@ def _init_posix(vars):
     # _sysconfigdata is generated at build time, see _generate_posix_vars()
     from _sysconfigdata import build_time_vars
     vars.update(build_time_vars)
+    # On AIX, we have different paths for building the Python modules
+    # relative to the Python source, and building third party modules
+    # after installing the Python dist.
+    if _PYTHON_BUILD:
+        vars['LDSHARED'] = vars['BLDSHARED']
 
 def _init_non_posix(vars):
     """Initialize the module as appropriate for NT"""
msg230257 - (view) Author: Phil Elson (pelson) * Date: 2014-10-30 09:38
FWIW - this doesn't appear to have been backported to v2.7.x. As a result, the ./Modules/ld_so_aix reference still exists in _sysconfigdata.py in v2.7.8 (and therefore I was unable to build numpy). The workaround is easy enough, it is just something that is worth noting.
History
Date User Action Args
2014-10-30 09:38:19pelsonsetnosy: + pelson
messages: + msg230257
2014-06-06 15:23:23haubisetmessages: + msg219888
2014-06-03 08:42:54haubisetmessages: + msg219670
2014-06-03 08:41:01haubisetfiles: - python-tip-aix-absbuilddir.patch
2013-12-05 17:57:07haubisetmessages: + msg205315
2013-12-05 16:26:26David.Edelsohnsetmessages: + msg205310
2013-12-05 13:51:27haubisetmessages: + msg205297
2013-11-08 16:46:39David.Edelsohnsetmessages: + msg202436
2013-11-08 15:24:30haubisetfiles: + python-tip-aix-absbuilddir.patch
nosy: + haubi
messages: + msg202427

2013-10-22 04:18:56David.Edelsohnsetmessages: + msg200875
2013-10-19 20:08:38pitrousetstatus: open -> closed
resolution: fixed
messages: + msg200492

stage: resolved
2013-10-19 20:07:54python-devsetnosy: + python-dev
messages: + msg200491
2013-10-19 20:00:13pitrousetnosy: + pitrou
messages: + msg200488
2013-10-19 19:02:44David.Edelsohnsetmessages: + msg200479
2013-10-19 18:48:19David.Edelsohnsetfiles: + issue18235.patch
keywords: + patch
messages: + msg200476
2013-10-19 18:39:55larrysetnosy: + larry
2013-06-22 00:35:37David.Edelsohnsetmessages: + msg191624
2013-06-21 17:15:22eric.araujosetversions: + Python 3.3, Python 3.4, - Python 3.5
nosy: + eric.araujo

messages: + msg191598

components: + Build, - Extension Modules
2013-06-18 01:26:22David.Edelsohnsetmessages: + msg191385
2013-06-17 00:33:21David.Edelsohncreate