This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Resolve distutils option conflicts with priorities
Type: Stage: resolved
Components: Distutils Versions: Python 3.4, Python 3.5, Python 2.7
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: dstufft, eric.araujo, lkollar, minrk, nchammas, piotr.dobrogost, pitrou, ssbarnea, steve.dower, tdsmith
Priority: normal Keywords: patch

Created on 2014-08-24 22:32 by minrk, last changed 2022-04-11 14:58 by admin. This issue is now closed.

File name Uploaded Description Edit
distutils_conflict.patch minrk, 2014-08-24 22:32 add priority-based distutils option conflict resolution
Messages (7)
msg225843 - (view) Author: Min RK (minrk) * Date: 2014-08-24 22:32

Some Python distros (OS X, Debian, Homebrew, others) want the default installation prefix for packages to differ from sys.prefix. OS X and Debian accomplish this by patching distutils itself, with special cases like `if sys.prefix == '/System/Library/...': actually_do_something_else()`. Homebrew accomplishes this by writing a `distutils.cfg` with:

    prefix = /usr/local

The distutils.cfg approach is certainly simpler than shipping a patch, but has its own problems, because distutils doesn't differentiate the *source* of configuration options when resolving conflicts. That means that you can't do `python install --user` because it fails with the error "can't combine user with prefix, ..." without also specifying `--prefix=''` to eliminate the conflict.


I've included a patch for discussion, which uses the fact that the option_dict tracks the source of each option, and keeps track of the load order of each. In the case of an option conflict, the option that came from the lower priority source is unset back to None. If they come from the same source, then the same conflict error message is displayed as before.

Even if this patch is rejected as madness, as I expect it might be, official  recommendations on how to address the root question of `sys.prefix != install_prefix` would be appreciated.
msg225888 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-25 18:46
Hmm... if it's only about --prefix vs. --user, we could let --user take precedence and issue a warning instead of an error.
msg226004 - (view) Author: Min RK (minrk) * Date: 2014-08-27 19:13
`--prefix` vs `--user` is the only conflict I have encountered, but based on the way it works, it could just as easily happen with any of the various other conflicting options in install (install_base, exec_prefix, etc.), though that might not be very common.

There is a general question: If a Python distributor wants sys.prefix and default install_prefix to differ, what's the right way to do it? Setting it in distutils.cfg makes sense other than the conflicting option issues. Could there be a special `default_prefix` key that gets used as the final fallback (end of install.finalize_unix)?

I would really like to avoid having a warning on every install, since warning suggests that something has been done incorrectly, which in turn suggests that `distutils.cfg` is the wrong place to set the install prefix.
msg230992 - (view) Author: Tim Smith (tdsmith) * Date: 2014-11-11 00:11
Ping! Any chance for feedback here? This behavior took me by surprise again today. :)
msg233085 - (view) Author: Sorin Sbarnea (ssbarnea) * Date: 2014-12-24 18:47
Can't we do something about this bug? I think that most python devs are ignoring the severity of this one, especially because it does break distribution of packages (or at least their testing)

On OS X the experience is awful as --user doesn't work at all and as a maintainer of many python packages I end-up with a case where installing packages for the current user doesn't work. It works well on all other platforms but not on OS X (when you have the brew python). 

We cannot really expect all the package developers to spend a lot of time creating new scenarios only for trying to bypass this bug.

For the sake of our Python users let's fix this and make the Python experience a good one, especially for people that do not have much experience with python.

Maybe Santa will get my message ;) Wish you happy holidays!
msg233124 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-12-27 00:04
I do think this needs fixing. It's just that the patch is non-trivial and someone must take the time to review it.
Also, the patch should add a unit test to avoid regressions.
msg386299 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:16
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at
Date User Action Args
2022-04-11 14:58:07adminsetgithub: 66465
2021-02-03 18:16:19steve.dowersetstatus: open -> closed

nosy: + steve.dower
messages: + msg386299

resolution: out of date
stage: patch review -> resolved
2019-04-24 20:23:13lkollarsetnosy: + lkollar
2018-05-13 16:42:37nchammassetnosy: + nchammas
2015-04-17 05:55:42piotr.dobrogostsetnosy: + piotr.dobrogost
2014-12-27 00:04:50pitrousetmessages: + msg233124
2014-12-24 18:47:23ssbarneasetnosy: + ssbarnea
messages: + msg233085
2014-11-11 00:11:57tdsmithsetmessages: + msg230992
2014-08-29 20:38:10terry.reedysetstage: patch review
versions: - Python 3.1, Python 3.2, Python 3.3
2014-08-27 19:13:13minrksetmessages: + msg226004
2014-08-25 18:46:57pitrousetnosy: + pitrou
messages: + msg225888
2014-08-25 17:09:05tdsmithsetnosy: + tdsmith
2014-08-24 22:32:24minrkcreate