New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Define a standard location and API for configuration files #51424
Comments
[.]pydistutils.cfg will be deprecated in favor of a single |
Why is this? .local/ contains machine-installed files while |
That's what is returned by site.getuserbase() (which uses PYTHONUSERBASE, and defaults to ~/.local) PYTHONUSERBASE is the root of Python user-specific paths so it makes This came up with the discussion we had with others at Distutils-SIG read this thread here for more info: [.]pypirc should probably go there too. |
Well, that makes it the user-specific equivalent of /usr or /usr/local.
That doesn't make sense actually, since .local isn't Python-specific. :) |
I've read the thread now and I think the original proposal of having a |
+1 to what Antoine said. |
The distutils.cfg file is located in the distutils package itself ! So the target is ~/.local/pythonX.X/site-packages/distutils/distutils.cfg (sorry if that was unclear) Now, I would totally agree to have a better ~/.NAME to reunite |
Ah, that's horrible. -1 to that. |
I like ~/.python very much too (that was my initial proposal IIRC). I guess the best way to add this support is to provide a new api in Distutils can move pypirc and distutils.cfg file there. Now for the global distutils.cfg, if we want to have it somewhere else I don't know though how the latter would look under windows |
Tarek Ziadé wrote:
+1 on that name as well. One thing to note though: it is well possible that a user uses
I think Subversion get's this pretty right, so we might want http://svnbook.red-bean.com/en/1.1/ch07.html#svn-ch-7-sect-1 |
The subversion system looks pretty nice ! So here's a modified proposal that covers the python version as well. = per user configuration files = ~/.pythonMAJOR.MINOR Examples: ~/.python2.7 On Windows, ~ will be replaced by APPDATA if founded in the environment. == global configuration files == On Linux/Mac: /etc/pythonMAJOR.MINOR examples: /etc/python2.7/ On Windows: \All users\Application Data\PythonMAJORMINOR (\All users\Application Data is specified by the Windows Registry) examples: \All users\Application Data\Python26 == standard naming for configuration files == The files are all named "NAME.cfg", in all platforms. == what gets in per-user configuration directory ==
== what gets in global configuration directory ==
== apis added == in site.py:
(the names are following site.py naming style) |
FYI, /etc/pythonX.Y (and /etc/python) are used by Debian; the former I can't see how this could become a problem, but maybe someone else does. |
Thanks for the info Georges. I am adding Matthias and Michael so they can give us their opinions for |
Personally I'm uncomfortable with creating As we now have is ~/.local/pythonX.Y I would reuse this. |
But ~/.local is simply not the location for config files. |
For Linux and Mac OS X put the config file wherever the 'right' place is On Windows we create: %APPDATA%/Python/PythonX.Y/site-packages If it is to no longer live inside distutils (which seems reasonable) On Windows there is no 'right' place, the officially right place for the |
Noting of course that on IronPython it should be: or wherever we decided in the end. :-) |
/etc is definitely not the right place to put files for OS X framework (Adding Ronald.) |
Ned Deily wrote:
I think there is a misunderstanding here: we're trying to find AFAIK, there is general agreement that when writing files, OTOH, searching for config files that the user creates and |
I don't think there's a misunderstanding. By "putting", I meant "reading" |
I agree with Ned that neither ~/.local nor /etc are a good fit for OSX, The common unix directories are often not a good pick for good OSX The OSX filesystem structure should be described somewhere below Technically configuration files should be stored in /Library/Preferences BTW. I haven't read most of the discussion yet, I'll probably have more |
When ~/.local/lib was chosen for the user directory, the BaseDir Spec References: |
As a (mainly ex) windows user I would hate to have user editable data in APPDATA as it is not a location the user ever expects to visit. The home directory, or a subdirectory thereof, for user editable app specific data is more usual and more friendly. |
I guess I misremembered. %APPDATA% is /usr/share, right? What I meant was the variable that expands to the “documents and settings” directory. |
Pointed out by Tarek: http://github.com/ActiveState/appdirs#readme (shame it doesn‘t follow the BaseDir Spec on Free OSes). |
I still think that user editable configuration data, at least on windows, is different from application data. It would definitely seem weird to me to have a config file in the %APPDATA% directory. %APPDATA% would be fine for a cache or other application specific data. On Windows I would expect something like: "c:\Documents and Settings\AppName" (Note the settings in Documents and settings... - although in Windows 7 I think this might just be Documents now.) |
Yes, software built for /usr/local will often respect /usr/local/etc in addition to /etc. Also, many desktop Linux applications put state into ~/.local/share/$appname. I am not sure about "configuration files", (apps that use ~/.local will typically use ~/.config for configuration, but that is a different thing entirely) but autotools applications built with '--prefix ~/.local' will often respect ~/.local/etc.
I argued loudly for consistency in treatment of UNIX-like OSes when this decision was being made, and I stand by that position. Why would you want to break software that works fine on BSD and Linux by requiring that it specifically test for OS X, and do the exact same thing, but in a different directory? (See <http://en.wikipedia.org/wiki/Single_UNIX_Specification#Mac_OS_X_and_Mac_OS_X_Server\>) The addition of a per-user directory on OS X when there were no similar directories on other platforms was a weird oversight. The user site directory in Python 2.5 and earlier does not use any features of OS X and refers to ~/Library/Python only by convention. Plus, that convention didn't support the distutils properly anyway: if you put your library code into ~/Library/Python, where are you supposed to tell 'setup.py install' to put script files? With ~/.local, it's obvious: ~/.local/bin. The real OS X convention is to put your code _into your application bundle_, after all, not into random directories in ~/Library. If you don't have an application or framework bundle, you have one foot in UNIX-land already. The point of honoring platform conventions is to provide a consistent experience to users; some users on OS X will be expecting UNIX-like behavior, some will be expecting Framework-like behavior, and there is really no reason not to provide both. There is no reason not to consider /etc, for that matter. PHP, for example, still reads /etc/php.ini on OS X. |
It would be nice to settle this. Personally I would like to see the defaults being: Linux: ~/.pythonx.y For both Windows and Mac OS X I would be happy with the fallback / primary to be the other way round - it doesn't *really* matter. The API for getting the user config direction should always return a list I guess if we have fallbacks. Someone else in the Python-dev discussion thread (Adal Chiriliuc) expressed a preferences for the documents folder on Windows over the home directory. This would be fine as well (with AppData still as a fallback). This is used by other native windows applications. (In earlier versions of Windows the documents folder was explicitly called "Documents and Settings".) We should use pythonx.y rather than just python because (for example) you will typically have different packages installed when you have multiple versions of Python, and in unittest would then want / need different plugins enabled and configured in the unittest config files for each of the versions of Python. |
I agree with Michael Foord; my comment on bpo-8404 <http://bugs.python.org/issue8084#msg122935\> may be of interest to anyone looking at this as well. |
The issue proved more complicated than expected, due to antics of Mac OS X and Windows Vista, so I set it aside for later. I intend to summarize all useful comments from the python-dev thread and bug reports comments and try to reach agreement for 3.3. |
I'm mildly uncomfortable advocating many different new dot directories in $HOME (e.g. .python2.7 .python3.1 .python3.2). Let's say -0. Also, because of backward compatibility, I think most configuration files will end up being similar enough between Python versions. Without having thought through all the details, it seems that ~/.python would work in general for configuration files that are compatible across Pythons, with X.Y subdirectories for version specific config files. |
I would still prefer ~/.local/something-parallel-to-where-it's-usually-installed for its ease of documentation. But ~/.python/X.Y isn't terribly objectionable. A minor point about Michael Foord's fallback proposal: I don't know why everyone is so keen to stuff things into ~/Library on the mac, but dotfiles don't go in ~/Library/Preferences. While that directory isn't exclusively populated with plist files, it's pretty close. If you want to try for consistency with platform convention, 'org.python.distutils.cfg' would be a better name for it, but I've never hand-edited a text file in that directory either and I get the impression one isn't meant to. But, since framework builds of Python already has ~/Library/Python/X.Y directories that they look in for other things, that location should probably be recycled if one wishes to have a fallback. |
See also bpo-12393. |
no, sorry, but ~/.python is wrong on linux. there exists a standard about where things belong: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html it says that configuration stuff belongs into i propose to add the |
That standard is just wrong, dotfiles belong in $HOME :) I'm only half joking, having grown up with Unix and having had the experience of flailing around for a while before I figured out a particular program's rc file was in .config instead of where I expected it to be, but I *am* half joking, because putting them all in .config is convenient. But most programs don't, so also looking in ~ makes sense to me. |
I'm of mixed opinion. I personally don't like cluttering up my $HOME with a |
If this is about existing config files, then we should make sure we don't break existing installations. Also, multiple possible locations in the home dir are confusing. If this is about new config files, then ~/.config/python/XXX is probably ok. I'm not sure what XDG_CONFIG_HOME is but $XDG_CONFIG_HOME doesn't seem defined under my shell (Xubuntu 15.04). |
XDG_CONFIG_HOME is not generally set to anything; the default of ~/.config is usually fine. However, the point is that you _can_ set it to point at a different location. The relevant specification is here: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html The biggest reason I strongly believe we should follow a specification is that Python is not the only software on the system. If I am an ops person that has to deal with Python and Ruby and Java and C software all the time, *any* consistency is a big help. We don't have to follow *this* specification, but we should follow *some* specification (although this specific one looks like a good one to me). The Python community has no special expertise deciding on locations for configuration files, and so we should not be trying to invent a new and better place for them. There are other advantages to following the XDG spec. If we follow it correctly (and not, like Barry suggested, start adding random other directories like ~/.python), users can easily switch between configuration environments by switching the XDG environment variables (both XDG_CONFIG_DIRS and XDG_CONFIG_HOME), which would be a handy way of ignoring both user-specified config files _and_ system-specified ones, to get a pristine virtualenv-like separation between different config files. Given that they are going to need to set these env vars anyway to redirect spec-compliant libraries that their application may be using, it would be nice to just have _one_ set of meta-configuration variables rather than one for Python and one for C and one for Ruby and so on. Consider the fact that distutils respects "CFLAGS", and did not feel the need to invent "DISTUTILS_FLAGS_FOR_THE_C_COMPILER". |
On Sep 01, 2015, at 07:15 PM, Glyph Lefkowitz wrote:
I was really just suggesting reading from other locations for backward |
of course if there is a chance that some specific config file exists at a known location, it’s the only sane choice to try and look for it there iff it isn’t in its preferred location. e.g. fontconfig made that switch some time ago: it used to be ~/.fonts.conf, now it’s ${XDG_CONFIG_HOME-$HOME/.config}/fontconfig/fonts.conf the library will look at the new position and if it can’t find anything there, it looks at the old one. but for new config files we should follow standards. on windows and OSX there are different ones: see the current discussion on python-ideas for specifics and problems. but on linux the only standard for this which i know is XDG-base-directories, so we should follow that for new config files. |
The XDG standard seems to focus on desktop GUI applications and that's also where it's mostly used. Python has it's own installation scheme, which is documented at the top of sysconfig.py and which we've had ever since distutils became part of the stdlib. For user installs, this bases all installations under the PYTHONUSERBASE dir, which defaults to ~/.local/ on Unix. So user installs go into ~/.local/pythonX.X/ (I also have a ~/.local/python/X.X/ - not sure where that came from). Python also uses ~/.pypirc for PyPI, ~/.python-eggs for setuptools, ~/.python_history for readline history. The installation scheme currently doesn't define a dir for config files, only for "data" files, so adding one would probably be wise. Since we already own ~/.local/pythonX.X/, why not put config files in there ? For some extra Unix touch, we could use ~/.local/pythonX.X/etc/ and then move the cache bits to a ./var/ subdir (by defining another new "cache" entry for the schemes). |
hi mark, i’ve just lengthily replied to you on python-ideas. in short: nope! many command line tools that are relatively new (among them your examples git and pip) honor the specs, my ~/.cache, ~/.config, and ~/.local/share is full of things belonging to libraries, cli tools, my DE, and of course also GUI applications. please let’s continue this discussion on python-ideas if you’re still not convinced ☺ |
On 02.09.2015 16:37, flying sheep wrote:
I think we're mixing two discussions here:
For 1. we already have a solution (in sysconfig.py). If you want to For 2. we can use whatever convention or standard is en vogue today. Note that this ticket started with the location of the distutils Given the experience with the platform module, I'm not convinced |
you’re right about the two problems being mixed, however not about the standards. also you’re intermingling conventions with actual standards. the directory standards for all three big OSs are either not going to change or fitted with a backwards-compatibility layer (like windows did with the “My Music/Videos/*” folders): http://windows.microsoft.com/de-de/windows-8/what-appdata-folder https://msdn.microsoft.com/en-us/library/windows/desktop/dd378457%28v=vs.85%29.aspx http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html all three OSs have stable, widely followed standards in place, and the idea of providing a python stdlib API for them received an almost unanimously positive response – with the sole exception being you. |
On 03.09.2015 13:37, flying sheep wrote:
I'm not against developing another module based on appdirs for this. I do question the usefulness of having such a module in the stdlib, Regarding standard vs. convention: that's all smoke and mirrors :-) But this ticket is not about that discussion anyway... |
As someone else mentioned, the freedesktop standard is only really "widely followed" by desktop aps. My own .config directory contains 8 entries, while there are 13 rc files/directories in my homedir (one of which is my X windows manager :) That's not to mention the 50+ config files in my home directory that don't end with 'rc'.... |
just because people do something doesn’t mean it’s right. i guess the tendency for CLI applications to do it wrong comes from
on the other hand, GUI applications usually have a build system in place for UI file compilation, need a .desktop file installed, and so on, i.e. need more boilerplate and dependencies from the start, and people have to research how to do things properly. GUI libraries have their built-in standard dirs interfaces as well. ———————— another thing. on the python-ideas thread someone mentioned that
this is horrible! this is so obviously the wrong thing to do. i hope having the module will reduce the number of similar atrocities. |
The "tendency of CLI applications to do it wrong" comes from the fact that they are following the *older* unix (de-facto) standard, which is to put config files in the home directory as dot files. That is, they are *not* doing it wrong, they are following the older unix de-facto standard and not the freedesktop.org standard (which, you will note, is a standard arising out of GUI applications, not CLI applications). Not, mind, you, that I think .config is a bad standard, I'm just saying that if you want to follow standards you need to account for the legacy standard as well as the new standard. |
This ticket did not reach a consensus. It seems there would be little value in changing the existing scheme for IDLE and such; external tools can (and often do) already use appdirs; I suggest closing this. |
The early discussion was about distutils, which is gone. Mark-Andre Lemburg noted that the expanded scope is too broad for an issue ('ticket') and recommended closing. 'merwok' also recommended closing. I agree. |
Latest discussion: https://discuss.python.org/t/add-cross-platform-user-directories-to-pathlib-standard-library/45390 (also without clear consensus) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: