Author eric.araujo
Recipients alexis, eric.araujo, erik.bray, jkloth, paul.moore, tarek
Date 2011-11-15.17:27:29
SpamBayes Score 1.11022e-16
Marked as misclassified No
Message-id <1321378051.04.0.391182679832.issue11805@psf.upfronthosting.co.za>
In-reply-to
Content
> Looking at sysconfig.cfg makes it all pretty clear, though it was hard to find this
> information a while ago.
Yeah, you had to be here when the resources code was committed, or to stumble on it while reading the source code.  It’s one of the big things I will document as part of #12779.

> So what exactly is the verdict then?  Keep package_data for now, but encourage resources,
> and dump data_files all together?  Or...?
The reverse: resources are the new way of installing data_files.  package_data is obsoleted by that system and removed.

> But as far as I can see, there's no way in packaging to describe a module that works like sysconfig
> itself! (A module with a resource which is expected to live alongside the .py file).
For bootstrapping reasons, sysconfig.cfg needs to be the one exception to the rule.  (Yes, I think about moving distutils.cfg and wininst-*.exe files according to sysconfig —and pydoc data and venv shell scripts!  I will make sure we reach agreement on python-dev first, of course :)

The point is that it is not possible to have a data file living alongside the py file.  It is by design.  The new system caters to downstream packagers while trying not to be bothersome to Python authors.  Why do you care where your data file is installed, as long as your code can get it?

There is one argument against this system: It means that your code has a runtime dependency on a function provided by the packaging tool.  This is one of my griefs against setuptools.  The difference here is that the dist-info dir is an accepted PEP and packaging is in the stdlib, and that the dependency is minimal (it does not make you learn a parallel import system for example).  However, one drawback is that we expect and support a transition period where people have parallel setup.cfg and setup.py (with the setup.py possibly taking its info from setup.cfg), so adding support for d2 will not only require adding a setup.cfg and modularizing complex setup.py into hooks: it will require code editions to have code branches for database.get_file and __file__ + open.  That won’t look good.

> Installing the cfg file to {purelib} would work in practice, but if the distribution containing
> sysconfig was later changed to include a C extension, then the code would get silently switched to
> install into platlib, and the .cfg file would no longer be alongside the .py file.
> 
> (This is only actually the case when purebase and base differ - which never happens on Windows, and
> I don't really understand when it would happen on Unix. So I can't really say how likely and/or
> important this possibility is. But it's certainly there.)
There is a distinction between prefix and exec-prefix.  I think it may come from a time where you’d have one tree served for many machines by an NFS server and architecture-specific trees on a local filesystem.

> [...]
> OK, having said all that, I do think that saying "package_data was wrong, let’s move away from that"
> is a bit user-unfriendly. Whether it causes issues for OS distributors, or is "wrong" as a matter of
> principle, people do use it, a lot.
Because they had no alternative: you couldn’t predict where data_files would end up, so package_data + __file__ was the universal kludge.

> It's the basis of the whole egg concept (a package and its data as a single self-contained object)
distutils2 does not support the egg concept.  In a post-PEP 376 world, an installed distribution can be one or more modules with a dist-info directory, one or more packages with a dist-info directory, etc.

> If you don't support it, I predict that people are simply going to invent more and more elaborate
> hacks to emulate it.
Interesting.  I have to go now, but be sure I’ll think over this, bring it up on pydev and make sure the situation is acceptable before Py 3.3b1 / d2 1.0b1.
History
Date User Action Args
2011-11-15 17:27:31eric.araujosetrecipients: + eric.araujo, paul.moore, tarek, jkloth, alexis, erik.bray
2011-11-15 17:27:31eric.araujosetmessageid: <1321378051.04.0.391182679832.issue11805@psf.upfronthosting.co.za>
2011-11-15 17:27:30eric.araujolinkissue11805 messages
2011-11-15 17:27:29eric.araujocreate