[From: http://bugs.debian.org/134762]

FHS Compliance - .py{,c} are architecture independant
thus belong in /usr/share

The Python manual makes it clear that byte compiled
python files are
 platform independant, and thus belong in
arch-independant packages and stored in /usr/share, as
per the FHS recommendations for such things.

So the request is to store them in
<prefix>/share/pythonX.Y.

Logged In: YES 
user_id=60903

FHS: Filesystem Hierarchy Standard

    http://www.pathname.com/fhs/

Logged In: YES 
user_id=21627

I think this requires a PEP. A Python package can consist of
byte code modules and extension modules; arranging the
package loader to find those in different directories is a
challenge.

Logged In: YES 
user_id=163326

Python runs on dozens of platforms besides Debian Linux.
Thus the Linux FHS shouldn't concern Python at all. I'd
propose to close this bug as "Invalid".

Logged In: YES 
user_id=60903

The reason given to close the report seems to be invalid.
The FHS has nothing to with Debian (except that we follow
the FHS). The FHS is targeted at Unix distributions, so it's
neither limited to a single distribution nor to Linux
distributions in general.

Logged In: YES 
user_id=45365

I'm confused. If you configure with --exec-prefix=/foo --prefix=/foo/share/pythonX.Y isn't that good enough?

If it's good enough (i.e. if it allows you to build a Python that adheres to the FHS if you are so inclined) that I agree with ghaering: there's no reason to force people to adhere to the Filesystem Hierarchy Standard, so let's close the bug.

If it is impossible to make an FHS-compliant distribution with the current setup: please explain.

Logged In: YES 
user_id=60903

Not yet.

--prefix=/foo/share/pythonX.Y would lead to
/foo/share/pythonX.Y/lib/pythonX.Y.

The SCRIPTDIR is somewhat hardcoded in getpath.c. So it's
not possible
to install into /foo/share/pythonX.Y, only
/foo/share/lib/pythonX.Y is supported.

The FHS doesn't specify where to put files inside
/usr/share, but most distributions put application specific
files directly in /usr/share.

Logged In: YES 
user_id=60903

when configured with --prefix=/usr/share and
--exec-prefix=/usr, python installs the header files into
/usr/share/include/pythonX.Y, which is at least unusual.
According to the FHS these files should go into
/usr/include/pythonX.Y

Logged In: YES 
user_id=45365

Well... the usual way in which people implemented sharing between 
architectures was that the /usr/share hierarchy duplicated the other 
hierarchies (i.e. with bin, lib, etc), and it was simply mounted cross-
platform from an nfs server. That's the architecture the Python install 
was created for. I have no idea why FHS modified this test-and-tried 
layout that's been in use for at least 15 years.

But: if you really want the other layout, why not submit a fix to 
configure.in and Makefile.pre.in? Simply add, say, --fhs-prefix=/usr/
share and if that option is present override the Makefile.pre.in 
declarations for SCRIPTDIR, LIBDEST and INCLUDEDIR?

(Hmm, coming to think of it: it seems rather silly that the FHS puts 
include files into /usr/include, where they aren't shared... If there's one 
thing that can be crossplatform it's source code....)

Logged In: YES 
user_id=45365

Well... the usual way in which people implemented sharing between 
architectures was that the /usr/share hierarchy duplicated the other 
hierarchies (i.e. with bin, lib, etc), and it was simply mounted cross-
platform from an nfs server. That's the architecture the Python install 
was created for. I have no idea why FHS modified this test-and-tried 
layout that's been in use for at least 15 years.

But: if you really want the other layout, why not submit a fix to 
configure.in and Makefile.pre.in? Simply add, say, --fhs-prefix=/usr/
share and if that option is present override the Makefile.pre.in 
declarations for SCRIPTDIR, LIBDEST and INCLUDEDIR?

(Hmm, coming to think of it: it seems rather silly that the FHS puts 
include files into /usr/include, where they aren't shared... If there's one 
thing that can be crossplatform it's source code....)

Logged In: YES 
user_id=163326

I assume that the Python directory layout is the same on all
currently supported platforms by Python. I really don't know
enough to be sure - the less that's true, the less my
following argument will be valid:

There are really two concerns:
1) make Python conform to the FHS
2) make Python behave the same cross-platform (Windows,
Unix, Mac, BeOS, OS/2, VMS, AS/400, ...)

You can't have both unless you force the FHS directory
layout onto all other platforms. I personally think that 2)
is a good thing.

I welcome the proposal of a configuration option to make
Python FHS-compliant, but I think it should not be the
default. Btw. you'd probably have to patch distutils, too.

Jack said that Pyhon include files should be cross-platform.
AFAIK they are, with one exceptions: pyconfig.h.

Logged In: YES 
user_id=163326

Sorry, Matthias. I was confusing the FHS with the Linux
Standard Base.

Logged In: YES 
user_id=45365

The layouts are similar, but not the same. MacPython-OS9 uses the source tree layout, and Windows-Python too, IIRC.
MacPython-OSX in a framework build uses a normal layout, but in a very funny place, with lots of symlinks to make it behave like a framework. I would be very surprised if, say WinCE python or AS/400 python had anything resembling a unix layout. But all these layouts are similar enough that I've never come across a Python where I felt completely lost.

I think there's nothing wrong with enabling people to use their preferred layout, if they have a good reason for it, but I would be against enforcing it unless the advantages are clear and universal.

And someone has to do the work:-)

Logged In: YES 
user_id=60903

Ok, I'll give --fhs-prefix a try. some questions:

- the lib_python in getpath.c hardcodes lib/pythonX.Y to 
search for the libraries. Is it enouogh to set PYTHONPATH to 
pythonX.Y?

- who to ask for distutils? are there concerns if a 
module/library is splitted over two directories? Or should there 
symlinks from /usr/lib/pythonX.Y to /usr/share/pythonX.Y?

- currently there is only one site-packages directory. how 
should two site-packages be supported (lib and share)?

- speaking of etc: this is a configuration file and should belong 
into the etc hierarchy. should etc be searched before or after 
the library directories?

Python's include files: not all packages separate platform 
specific headers from generic headers, probably therefore the 
FHS puts them in /usr/include.

Logged In: YES 
user_id=357491

Will PEP 304 solve your problem?

Logged In: YES 
user_id=60903

PEP 304 addresses one part: the location of the generated
.py[co] files. I fail to see, how it adds support to put .py
files in /usr/share.

So it partly solves the problem.

Logged In: YES 
user_id=357491

It won't help with that request.  Doing that would require changing install 
paths as suggested below.

As for your questions about implementing --fhs-prefix, I can answer a few.  
For Distutils questions you can just email python-dev to get the attention of 
Distutils developers.  For adding a second site-packages directory I am 
against (use PYTHONPATH if that is needed).

Logged In: YES 
user_id=209092

Did anyone bother *reading* the FHS?

http://www.pathname.com/fhs/2.2/fhs-4.7.html

> /usr/lib includes object files, libraries, and internal
binaries

http://www.pathname.com/fhs/2.2/fhs-4.11.html

> The /usr/share hierarchy is for all read-only architecture
independent data files.

.py{,c} files are *libraries*, not *data files*.

Thankyou, move along.

Logged In: YES 
user_id=357491

I think PYC files can be considered either libraries or data files.  
Either way I am making this a feature request instead of a bug.

Logged In: YES 
user_id=80475

It seems probable that no changes are forthcoming.  So, can 
this be closed?

Logged In: YES 
user_id=21627

As a feature request, this should be only closed if
implemented, definitely rejected (for all times, e.g.
because it is undesirable), or after being added to PEP 42.

Added to the PEP 42, rev 58638.

doko has closed the downstream bug on Thu, 19 Nov 2009, with the reason that .py{,c} are not data files in the FHS sense. (Note that even Lisp where code is data (and the reverse) is installed both under /usr/lib and /usr/share on Debian, so I don’t really know what to think.)

About the more general FHS problem, Tarek (the distutils and distutils2 maintainer) has a draft PEP about fine-grained control over every installation directory, so FHS layout for Python projects will be easily possible in the future. See http://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst (product of Pycon 2010, done with OS packaging experts, doko and other people). As for Python itself, we’ll see if there are other bugs about FHS compliance, but it seems good right now.