classification
Title: Use directories from configure rather than hardcoded
Type: behavior Stage: resolved
Components: Build, Distutils Versions: Python 3.2
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Problems with /usr/lib64 builds.
View: 1294959
Assigned To: tarek Nosy List: Arfrever, andybuckley, eric.araujo, loewis, matejcik, nijel, rothrock, rpetrov, sergiodj, soundmurderer, spektrum, tarek, vstinner
Priority: normal Keywords: patch

Created on 2003-12-12 10:24 by nijel, last changed 2020-02-12 00:53 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
Python-2.3.2-dirs.patch nijel, 2003-12-12 10:24
Messages (20)
msg45042 - (view) Author: Michal Čihař (nijel) Date: 2003-12-12 10:24
Current Makefile.pre.in contains paths like
$(exec_prefix)/lib, which bad especially for mixed
32-bit and 64-bit systems, where 64-bit things should
go to /lib64. Attached patch changes these paths to
@libdir@ (and does also simmilar changes for other
paths), which is usually outputed from configure.
msg45043 - (view) Author: Matejcik (spektrum) Date: 2006-09-11 17:44
Logged In: YES 
user_id=631694

this patch confuses the python 2.5 build process in a very
mysterious way - setup.py ends up using
-L/usr/lib/python2.5/config instead of -L. for
libpython2.5.so, which is obviously wrong.
msg45045 - (view) Author: Michal Čihař (nijel) Date: 2006-09-11 18:07
Logged In: YES 
user_id=192186

That doesn't surprise me, it was written in times of python
2.3...
msg45046 - (view) Author: Matejcik (spektrum) Date: 2006-09-11 20:12
Logged In: YES 
user_id=631694

well obviously, but if you look at the patch, there is
exactly -nothing- wrong with it.
actually, i'm surprised that it didn't get into 2.4 release.

but what's more, it should have no effect on setup.py at all
msg65368 - (view) Author: Sérgio Durigan Júnior (sergiodj) Date: 2008-04-11 19:48
Hi,

Continuing with my effort to improve Python's build system, I'd really
like to know why this issue has not been solved yet. I mean, apparently
this problem is still present in Python 2.5, since I can't change the
library's path with --libdir configure's flag. Any news about it?
msg73812 - (view) Author: Joseph Rothrock (rothrock) Date: 2008-09-25 20:38
Hi,

This problem still exists in 2.5.2. Setting the libdir argument doesn't
correctly set LIBDIR in the Makefile.


[rothrock@su1 Python-2.5.2]$ ./configure --prefix=/home/y
--libdir=/home/y/lib64

...

[rothrock@su1 Python-2.5.2]$ cat Makefile | grep LIBDIR=
LIBDIR=		$(exec_prefix)/lib
msg76707 - (view) Author: Roumen Petrov (rpetrov) * Date: 2008-12-01 21:41
The complete patch require redesign of python-distutils ;)
msg83076 - (view) Author: jan matejek (matejcik) * Date: 2009-03-03 17:34
well, seeing as redesign of distutils probably won't happen anytime soon...
why don't we fix the obvious bug (broken configure) now and redesign
distutils later?
msg83077 - (view) Author: jan matejek (matejcik) * Date: 2009-03-03 17:36
...the patch is absolutely trivial, doesn't have any deep consequences
and applies cleanly to head and all branches i know of.

...i wonder if the patch-fasttracking offer still applies...
msg87857 - (view) Author: (soundmurderer) Date: 2009-05-16 05:02
I can confirm that --bindir and --libdir are broken in Python 2.6.2,
despite the fact that ./configure --help lists them as helpful options.
 At the very least, ./configure should complain if you're using options
that don't do anything!  That would be the easy fix.

Here is what I am doing:

./configure \
    --prefix=/home/soundmurderer \
    --bindir=/home/soundmurderer/bin/x86_64/Python-2.6.2 \
    --libdir=/home/soundmurderer/lib/x86_64 

OK, now let's see the relevant section of the generated Makefile:

# Install prefix for architecture-independent files
prefix=         /home/soundmurderer/auzilov

# Install prefix for architecture-dependent files
exec_prefix=    ${prefix}

# Install prefix for data files
datarootdir=    ${prefix}/share

# Expanded directories
BINDIR=         $(exec_prefix)/bin
LIBDIR=         $(exec_prefix)/lib
MANDIR=         ${datarootdir}/man
INCLUDEDIR=     ${prefix}/include
CONFINCLUDEDIR= $(exec_prefix)/include
SCRIPTDIR=      $(prefix)/lib

# Detailed destination directories
BINLIBDEST=     $(LIBDIR)/python$(VERSION)
LIBDEST=        $(SCRIPTDIR)/python$(VERSION)
INCLUDEPY=      $(INCLUDEDIR)/python$(VERSION)
CONFINCLUDEPY=  $(CONFINCLUDEDIR)/python$(VERSION)
LIBP=           $(LIBDIR)/python$(VERSION)

So, the --libdir and --bindir flags had no effect on LIBDIR and BINDIR,
and I wind up editing the Makefile by hand to get desired behavior.
msg87858 - (view) Author: (soundmurderer) Date: 2009-05-16 05:56
I tried applying nijel's 2.3.2 patch to Python 2.6.2 Makefile.pre.in,
then configure/make/install with --bindir and --libdir flags to
./configure.  It works in terms of producing the correct Makefile with
LIBDIR and BINDIR that I want, but I get problems after the "make
install" step when I launch the Python interpreter.  This is what happens:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
'import site' failed; use -v for traceback
Python 2.6.2 (r262:71600, May 15 2009, 22:46:19)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

I can work around bad imports and "could not find libraries" by adding
various subdirs of my desired LIBDIR to PYTHONHOME and PYTHONPATH.  But
the point is -- I thought that a *properly* installed Python
distribution should know where to find its own standard libraries??  I
should not have to hack PYTHONHOME and PYTHONPATH to get my local
install to work... right?

Once upon a time in 2.5.x days, I was able to do a local install using
only ./configure --prefix, and that's it.  No hacking PYTHONHOME or
PYTHONPATH that I recall.

How is the path to stuff like site.py getting encoded in the Python
installation?  Because I suspect that although nijel's patch may work
for generating makefiles that put binaries/libs in the right place, that
encoding step is still getting botched, requiring to manually set
PYTHONHOME and PYTHONPATH as a workaround.

Or maybe I am just confused.
msg88065 - (view) Author: (soundmurderer) Date: 2009-05-19 05:42
OK, I am now convinced that even with this patch, the Python build
process still won't work correctly because --libdir (post-patch) will
not correctly set the default PYTHONHOME.  Moreover, the very way that
PYTHONHOME gets used needs to change.  PYTHONHOME specifies "prefix"
and/or "exec_prefix", but the problem is that to find the lib dir,
Python tacks "lib/pythonX.X" onto the ends of these paths!  More info here:

http://docs.python.org/using/unix.html#python-related-paths-and-files
http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME

So, even with this patch, I still can't install, for example, to:
/home/soundmurderer/lib/x86_64/python2.6

I am forced to install to a location that ends with "lib/python2.6", e.g.:

/home/soundmurderer/x86_64/lib/python2.6

or else hack some stuff together with symlinks to make PYTHONHOME go to
the right place.

This seems to be a problem more fundamental than just incomplete
"./configure" in the build process and I think it warrants another
ticket, which I'm about to open.

Now, in regards to --libdir, even though the patch makes the
installation put the files in the expected location, Python will
probably not find it.
msg88067 - (view) Author: (soundmurderer) Date: 2009-05-19 05:53
OK done, opened a feature request for this pet peeve of mine. ;-)  Maybe
someday I'll complain about something substantial...

See: http://bugs.python.org/issue6060
msg112209 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-07-31 21:17
FYI, disutils is being improved as a new package named distutils2. It is not a complete redesign, but we break backward compatibility when required, so there’s hope.
msg114578 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 22:25
About lib vs. lib64 vs. lib32, see also http://bugs.python.org/issue1294959#msg94941
msg114580 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 22:29
I don’t know much about Makefile.pre.in, so I can’t say anything about the patch. Martin, could you maybe comment?
msg114581 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 22:33
From http://bugs.python.org/issue1294959#msg94986

“The main problem is that Python's configuration system is not
geared up to having the lib directories for platform dependent
and platform independent parts use different names.

It currently only supports using different path *prefixes* for
such setups (--prefix and --exec-prefix), e.g. /usr and /usr64
would work just fine. It doesn't follow --libdir.”

So I don’t know whether there can be a simple fix for this or if we have to wait for the new sysconfig module (see http://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst)
msg114585 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-21 22:44
I don't think anything in distutils can solve whatever the problem is that people perceive.

All I can say about the patch is that it apparently doesn't do any harm. I expect that, if it is accepted, people continue complain that "it doesn't work". Unfortunately, Michal Čihař never indicated what *specific* problem this patch is supposed to solve, so it is difficult to tell whether it actually achieves that objective (the literal description "64-bit things should go to /lib64" is not achieved by the patch - it never mentions "lib64".
msg114715 - (view) Author: jan matejek (matejcik) * Date: 2010-08-22 22:57
the problem this patch was solving is "python libraries install into "prefix/lib" regardless of what is the correct local LIBDIR" - which is obviously broken on systems where LIBDIR is something other than "prefix/lib", most notable example being "prefix/lib64" on current 64bit linuxes.


but as far i can tell, this problem was fixed (to an extent) in 2.7 by partially applying this patch.
SCRIPTDIR is still prefix/lib, but the correctness of that is up for a long and winded debate.
msg361843 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-02-12 00:53
Start of this issue:

"Current Makefile.pre.in contains paths like $(exec_prefix)/lib, which bad especially for mixed 32-bit and 64-bit systems, where 64-bit things should go to /lib64."

That's a duplicate of bpo-1294959. This issue discuss also modifying distutils but there is no clear plan on how it should behave or not. My PR 18381 of bpo-1294959 updates sysconfig and distutils to support $prefix/$python_libdir/ path.
History
Date User Action Args
2020-02-12 00:53:00vstinnersetstatus: open -> closed

superseder: Problems with /usr/lib64 builds.

nosy: + vstinner
messages: + msg361843
resolution: duplicate
stage: needs patch -> resolved
2011-03-16 13:55:16andybuckleysetnosy: + andybuckley
2010-08-22 22:57:33matejciksetmessages: + msg114715
2010-08-21 22:44:56loewissetmessages: + msg114585
2010-08-21 22:33:12eric.araujosetmessages: + msg114581
2010-08-21 22:29:23eric.araujosetnosy: + loewis
messages: + msg114580
2010-08-21 22:25:44eric.araujosetmessages: - msg45044
2010-08-21 22:25:15eric.araujosetmessages: + msg114578
2010-08-11 05:49:28eric.araujolinkissue1019715 superseder
2010-07-31 21:17:29eric.araujosetmessages: + msg112209
versions: + Python 3.2, - Python 2.6, Python 3.1
2010-05-15 23:46:07eric.araujosetmessages: - msg105838
2010-05-15 23:45:51eric.araujosetassignee: skrah -> tarek

messages: + msg105838
nosy: - skrah
2010-05-15 23:44:53eric.araujosetassignee: tarek -> skrah

nosy: + skrah, eric.araujo
2009-11-06 16:46:11Arfreversetnosy: + Arfrever
2009-05-19 23:38:14ajaksu2setnosy: + tarek
versions: + Python 3.1, - Python 2.5, Python 3.0, Python 2.7
assignee: tarek
components: + Distutils
type: behavior
stage: needs patch
2009-05-19 05:53:33soundmurderersetmessages: + msg88067
2009-05-19 05:43:34soundmurderersetmessages: + msg88065
2009-05-16 05:56:17soundmurderersetmessages: + msg87858
2009-05-16 05:02:34soundmurderersetnosy: + soundmurderer
messages: + msg87857
2009-03-03 17:36:02matejciksetmessages: + msg83077
2009-03-03 17:34:04matejciksetmessages: + msg83076
2009-03-03 16:20:49matejciksetnosy: + matejcik
2008-12-10 09:09:09loewissetversions: - Python 2.5.3
2008-12-01 21:41:13rpetrovsetnosy: + rpetrov
messages: + msg76707
2008-12-01 17:00:05akitadasetversions: + Python 2.6, Python 3.0, Python 2.7, Python 2.5.3
2008-09-25 20:38:48rothrocksetnosy: + rothrock
messages: + msg73812
versions: + Python 2.5, - Python 2.3
2008-04-11 19:49:00sergiodjsetnosy: + sergiodj
messages: + msg65368
2003-12-12 10:24:30nijelcreate