classification
Title: Python 3.3/3.4 installation issue on OpenSUSE lib/lib64 folders
Type: Stage: patch review
Components: Build Versions: Python 3.4, Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, catalin.iacob, doko, eric.araujo, ita1024, nathanr, rpetrov, skip.montanaro
Priority: high Keywords: patch

Created on 2012-08-12 17:58 by ita1024, last changed 2013-08-01 20:55 by catalin.iacob.

Files
File name Uploaded Description Edit
install_log.txt ita1024, 2012-08-13 22:30 install log
_sysconfigdata.py ita1024, 2012-08-13 22:31 Generatored sysconfigdata file
opensuse.diff skip.montanaro, 2012-10-06 03:00 review
Messages (14)
msg168059 - (view) Author: (ita1024) Date: 2012-08-12 17:58
On opensuse 12.1, python 3.3 installs its extensions in /usr/local/lib64/python3.3/lib-dynload/, but the .py files are in /usr/local/lib/python3.3/.

When launching python 3.3 one gets:
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python 3.3.0b1 (default, Aug 11 2012, 10:45:34) 
[GCC 4.6.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/etc/pythonstart", line 5, in <module>
    import atexit
  File "<frozen importlib._bootstrap>", line 1294, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1258, in _find_and_load_unlocked
ImportError: No module named 'atexit'

The same thing happens when installing with a --prefix.

Moving the directory lib64/python3.3/lib-dynload to lib/python3.3/lib-dynload fixes the problem.
msg168152 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2012-08-13 22:06
please could you attach the configure options, generated _sysconfigdata.py and a log of the install step?
msg168155 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2012-08-13 22:13
> File "/etc/pythonstart", line 5, in <module>

and this seems to be a patched/distro installation. Do you see this with an unpatched one as well?
msg168158 - (view) Author: (ita1024) Date: 2012-08-13 22:30
I have not modified anything related to python on my opensuse install, i have only grabbed the latest tarball, compiled and installed.

Here is the result with python 3.3.0 beta 2
./configure
make
(sudo make install) > log

I am observing the same outputs:
$ /usr/local/bin/python3.3
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python 3.3.0b2 (default, Aug 14 2012, 00:25:40) 
[GCC 4.6.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/etc/pythonstart", line 5, in <module>
    import atexit
ImportError: No module named 'atexit'
>>> import atexit
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'atexit'
>>>

The file /etc/pythonstart contains:

"""
# startup script for python to enable saving of interpreter history and
# enabling name completion

# import needed modules
import atexit
import os
import readline
import rlcompleter

# where is history saved
historyPath = os.path.expanduser("~/.pyhistory")

# handler for saving history
def save_history(historyPath=historyPath):
    import readline
    readline.write_history_file(historyPath)

# read history, if it exists
if os.path.exists(historyPath):
    readline.set_history_length(10000)
    readline.read_history_file(historyPath)

# register saving handler
atexit.register(save_history)

# enable completion
readline.parse_and_bind('tab: complete')

# cleanup
del os, atexit, readline, rlcompleter, save_history, historyPath
"""

I will attach _sysconfigdata.py next (only one attachment)
msg168159 - (view) Author: (ita1024) Date: 2012-08-13 22:31
Here is the generated Lib/_sysconfigdata.py file
msg168184 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2012-08-14 09:36
the configure step sets LIBDIR to /usr/local/lib64. Please find out why this is not set to /usr/local/lib.
msg171704 - (view) Author: Nathan Robertson (nathanr) Date: 2012-10-01 11:25
This is also an issue on openSUSE 12.2 with the release version of Python 3.3 when compiling from sources.

OBS (openSUSE Build Service) has RPMs for 3.3rc1. I'm assuming they've got a patch which fixes this issue, and looking at the spec file (lines 61, 62 - https://build.opensuse.org/package/view_file?file=python3.spec&package=python3&project=devel%3Alanguages%3Apython%3AFactory&rev=cb7b94b33478cb8add5a5eb3ab3068dc):

# support lib-vs-lib64 distinction
Patch02:        Python-3.3.0b2-multilib.patch

The URL for that patch is https://build.opensuse.org/package/view_file?file=Python-3.3.0b2-multilib.patch&package=python3&project=devel%3Alanguages%3Apython%3AFactory&rev=cb7b94b33478cb8add5a5eb3ab3068dc

I haven't verified it, but I'm guessing that patch has something to do with the issue.

The top level directory for the sources for their RPMs are at https://build.opensuse.org/package/files?package=python3&project=devel%3Alanguages%3Apython%3AFactory
msg172164 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2012-10-06 03:00
I applied the OpenSUSE patch to the current cpython tip (3.4a0), rebuilt, then reinstalled.  I verified that sys.path contains directories which contain "lib64".  I can import the time module now, which failed before applying the patch.

The patch didn't apply perfectly (two chunks were slightly offset), so I generated a new patch, which is attached.

It's been awhile since I did anything with bug reports.  I tweaked a few fields.  Let me know if I muffed anything.
msg172180 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2012-10-06 11:05
the proposed patch has still some issues:

 - it breaks the installation on 64bit platforms on Debian and Ubuntu.
   Please test the patch on one of these platforms too.

 - it hardcodes more platform information in the sys modules, which
   makes it difficult to overwrite for cross builds. If these macros
   are needed then they should be taken from the sysconfig module,
   using the _sysconfigdata module.

 - use the host macros in the configure instead of uname

 - LIB shouldn't be necessary when configuring --with-libdir
msg172192 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2012-10-06 12:52
> the proposed patch has still some issues:
>
>  - it breaks the installation on 64bit platforms on Debian and Ubuntu.
>    Please test the patch on one of these platforms too.
>
>  - it hardcodes more platform information in the sys modules, which
>    makes it difficult to overwrite for cross builds. If these macros
>    are needed then they should be taken from the sysconfig module,
>    using the _sysconfigdata module.
>
>  - use the host macros in the configure instead of uname
>
>  - LIB shouldn't be necessary when configuring --with-libdir

Sorry, I don't have access to anything but the OpenSUSE systems at
work.  I was just verifying that it solved my installation problems.
I'll see if I can nudge some of the other things forward.
msg172304 - (view) Author: Roumen Petrov (rpetrov) Date: 2012-10-07 14:25
Skip,

You mix two technologies one is --libdir that could be specified at configure time and another one is sub-directory name for libraries as path suffix to some prefixes.

You could just adjust python to use user specified path (--libdir) and this is more flexible instead you hard coded stuff in configure script.
msg172307 - (view) Author: Roumen Petrov (rpetrov) Date: 2012-10-07 14:33
Also I could not understand why is opened new issue.

Just search for libdir and one is Issue 1294959  - 7 years old with the same idea. Later mean that solution is not acceptable.
msg172308 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2012-10-07 14:38
Before this goes any farther, let me make this clear - I *did not* develop this patch.  It was developed by the OpenSUSE folks.  I had trouble installing Python from source on the OpenSUSE system under my desk at work.  I asked about the problem on python-list@python.org.  Someone directed me to this open bug report.  I saw the reference to the patch, applied it, and installed.  I verified that the installed Python executable now works as I expected, and reported that.  That is all I did.  Various responses seem to suggest that people think I wrote the patch.  I did not.  I didn't even look at the contents of the patch before applying it.  I regenerated it relative to the current hg trunk simply because when I first applied it, a couple chunks were applied with offsets.

If my feedback is not sufficient to help move this bug report forward, my apologies.  I know next to nothing about writing configure scripts, let alone the right way to do that.  I don't mess around at this level of things any more.  Haven't in at least a decade.  I don't know what a host macro is.  I have never used --libdir.  If people are looking to me to correct the flaws in this patch, they are likely going to wait for awhile.
msg194118 - (view) Author: Catalin Iacob (catalin.iacob) * Date: 2013-08-01 20:55
In reply to msg168184, LIBDIR is set to include lib64 instead of lib because openSUSE explicitly does it that way in their multilib implementation.

More specifically, the CONFIG_SITE environment variable is set to /usr/share/site/x86_64-unknown-linux-gnu which contains, among more stuff:

catalin@opensuse:~/hacking/cpython> cat $CONFIG_SITE  | grep libdir
# If user did not specify libdir, guess the correct target:
if test "$libdir" = '${exec_prefix}/lib' ; then
                libdir='${exec_prefix}/lib64'


/usr/share/site/x86_64-unknown-linux-gnu is owned by package site-config whose README says:

site-config: Site Paths Configuration for autoconf Based configure Scripts
==========================================================================

Site configuration for autoconf based configure scripts provides smart
defaults for paths that are not specified.

All autoconf based configure scripts will automatically resource site
script using CONFIG_SITE environment variable. It works without any
explicit user interaction.

Currently implemented features:


Automatic libdir setup to $exec_prefix/lib or $exec_prefix/lib64
----------------------------------------------------------------

Depending on architecture, site script should correctly and
automatically switch between lib and lib64 libdir.


libexecdir setup to $exec_prefix/lib
------------------------------------

Upstream libexecdir defaults to $exec_prefix/libexec. This directory is
not part of FHS 2.2, so we change it to $exec_prefix/lib (yes, it is
correct to set it to $exec_prefix/lib even for bi-arch platforms). Most
projects add package name to this path, so you most probably get what
FHS 2.2 expects.
History
Date User Action Args
2013-08-01 20:55:41catalin.iacobsetnosy: + catalin.iacob
messages: + msg194118
2012-10-07 14:38:15skip.montanarosetmessages: + msg172308
2012-10-07 14:33:58rpetrovsetmessages: + msg172307
2012-10-07 14:25:02rpetrovsetnosy: + rpetrov
messages: + msg172304
2012-10-06 12:52:54skip.montanarosetmessages: + msg172192
title: Python 3.3/3.4 installation issue on OpenSUSE lib/lib64 folders -> Python 3.3/3.4 installation issue on OpenSUSE lib/lib64 folders
2012-10-06 11:05:53dokosetmessages: + msg172180
2012-10-06 03:01:02skip.montanarosetfiles: + opensuse.diff


title: Python 3.3 beta 1 installation issue lib/lib64 folders -> Python 3.3/3.4 installation issue on OpenSUSE lib/lib64 folders
keywords: + patch
nosy: + skip.montanaro
versions: + Python 3.4
messages: + msg172164
stage: patch review
2012-10-01 11:25:29nathanrsetnosy: + nathanr
messages: + msg171704
2012-08-17 18:12:43eric.araujosetnosy: + eric.araujo
2012-08-14 09:36:35dokosetmessages: + msg168184
2012-08-13 22:31:23ita1024setfiles: + _sysconfigdata.py

messages: + msg168159
2012-08-13 22:30:33ita1024setfiles: + install_log.txt

messages: + msg168158
2012-08-13 22:13:15dokosetmessages: + msg168155
2012-08-13 22:06:26dokosetmessages: + msg168152
2012-08-13 21:10:56georg.brandlsetnosy: + doko
2012-08-12 19:52:08pitrousetpriority: normal -> high
2012-08-12 18:22:39Arfreversetnosy: + Arfrever
2012-08-12 17:58:23ita1024create