classification
Title: win_add2path.py sets wrong user path
Type: behavior Stage:
Components: Demos and Tools, Windows Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Emil.Styrke, cheryl.sabella, christian.heimes, iritkatriel, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2012-10-26 12:41 by Emil.Styrke, last changed 2021-06-20 17:30 by iritkatriel.

Files
File name Uploaded Description Edit
addpath.patch paul.moore, 2015-03-23 19:38 review
Messages (19)
msg173840 - (view) Author: Emil Styrke (Emil.Styrke) Date: 2012-10-26 12:41
OS: Windows 7 Ultimate x64
Python version: 2.7.3 x64

win_add2path.py in the scripts directory is supposed to add the Scripts directory according to its source.  However, it tries to add $PYTHONPATH/Scripts, when in fact the Scripts directory is at $PYTHONPATH/Tools/Scripts in 2.7.

However, it seems like e.g. Setuptools still installs itself in $PYTHONPATH/Scripts, so maybe the right solution is to add both directories.

Furthermore, the script sets the default user path to %PATH%, which is not correct at least on windows 7.  I have to manually delete this part of the user path to get the user path merged correctly with the system path in cmd.exe.
msg173842 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2012-10-26 13:26
[this response appears to have got lost in an email black hole somewhere]

I assume that "$PYTHONPATH" is actually referring to sys.exec_prefix
(and not the PYTHONPATH env var which has nothing to do with this).

In any case c:\python27\scripts is still the place for scripts. There is
a c:\python27\tools\scripts (which you're welcome to add to your %PATH%
if you find the scripts useful) but that's not the place where
user-installed packages are expected to install their executable helper
scripts.

Without looking closely, I think I agree about default the user path to
%PATH%. I'll add Christian Heimes to this call who, I think, wrote the
original code.
msg173853 - (view) Author: Emil Styrke (Emil.Styrke) Date: 2012-10-26 15:28
Ok, thanks for clearing that up.  And yes, I meant the pythonpath variable in the script, which seems to be derived from sys.executable.

But in that case, it's unfortunate that the c:\python27\scripts dir is not created during install.  I did win_add2path right after install, and then it didn't add the scripts path because it didn't exist.
msg224318 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-30 15:46
Should we create the scripts dir during install?  Is it that big a deal?
msg238966 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-03-23 01:06
Presumably we can look at this after the discussions about the installer scheduled for PyCon 2015?
msg239043 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-03-23 16:40
It looks like the script hasn't been updated for the moved user scripts directory (under %APPDATA%), but otherwise this is not a bug and can be closed.
msg239044 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 16:46
Pip and/or setuptools will add the Scripts directory when needed, so it's not a big deal that the installer doesn't create it. In 2.7.9 and later, and 3.4 onwards, the Scripts directory is created as part of the ensurepip step of the install, so it's pretty much a moot point anyway.
msg239050 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-03-23 19:05
This script adds them to PATH. As I said, the original issue is not a bug, but it has drawn attention to something we apparently missed a while ago.
msg239051 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 19:21
Steve - is it the fact that it's not using the versioned user-site directory that you're referring to?
msg239052 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 19:22
... because I think it does do that (see http://bugs.python.org/file38085/userscripts.patch which updates this file)
msg239053 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 19:30
I think the problem here is that as the OP mentioned, win_add2path.py doesn't add the directory if it doesn't exist, so if you run it immediately after install, it won't add the directories that don't exist yet. And because it's setting the registry entries to make the path changes persistent, you only run it once and don't rerun it after those directories are created. Although if you did, that would fix the problem...

Maybe win_add2path.py should add the directories even if they don't exist? It only means removing the isdir check from the line

    if path and path not in envpath and os.path.isdir(path):

I don't really have a feel for whether that would be a good idea. I guess the only harm would be a bit of clutter on %PATH%, and probably no-one cares about that...
msg239054 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 19:32
One further thought - the installer can't create the user scripts directory if you're doing an all-users install, that's sort of the point of "all users". It could do so for a per-user install, but then there would be an inconsistency that I don't think is a good idea.

So maybe skipping the "does the directory exist" check is the only option.
msg239055 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 19:38
Here is a patch to fix the issue.
msg239057 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2015-03-23 20:09
Ah, you're right, it was fixed. Guess I misread my local copy (or I was looking at 3.4).

That patch looks good to me. It'll help cover the case where someone does something to create the directory.
msg239058 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 20:13
Cool. I'll try to set up a test (it's not covered by the testsuite AFAICT) on a VM or something, just to confirm I haven't broken anything.
msg239062 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 20:42
One further thought. This is a change in behaviour (albeit to a script in Tools), so I'm inclined to say it's a new feature for 3.5, rather than a bugfix to be backported to 2.7 and 3.4.

For users of 2.7/3.4, the workaround is simply to rerun the script - it doesn't add directories it has already added, so it's safe to rerun to add newly created directories.
msg239074 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2015-03-23 21:31
There's actually a bug in the pre-3.5 script beyond the "does the directory exist" check. The code is

    if hasattr(site, "USER_SITE"):
        userpath = site.USER_SITE.replace(appdata, "%APPDATA%")
        userscripts = os.path.join(userpath, "Scripts")

which sets userscripts as %APPDATA%\Python\Python34\site-packages\Scripts, which is completely wrong. Stripping off 2 directory levels gives the correct answer, but that seems a bit arbitrary - using sysconfig seems more robust.

Also, there's another point made in the original report - the registry variable HKCU\Environment\PATH is set to "%PATH%;<the stuff we add>". That's just wrong - the user PATH variable is added to the system PATH variable, so why interpolate the existing value? It may be that the behaviour varies in different Windows versions, I don't know enough about how older versions work to comment on that.

There are enough bugs here that I suspect that very few people have ever used the script :-( Maybe it would be better to leave the older versions alone, and simply rewrite or remove it for 3.5 (I know Steve has ideas about providing new path-setting scripts)
msg297620 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2017-07-03 23:41
Any interest in making a PR for this patch?
msg396184 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-06-20 17:30
I've refreshed the version as neither the patch was applied nor was the bug mentioned in msg239074 fixed so far.
History
Date User Action Args
2021-06-20 17:30:54iritkatrielsetnosy: + iritkatriel
messages: + msg396184
2021-06-20 17:27:30iritkatrielsetversions: + Python 3.11, - Python 2.7
2017-07-04 00:33:46BreamoreBoysetnosy: - BreamoreBoy
2017-07-03 23:41:17cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg297620
2015-03-23 21:31:25paul.mooresetmessages: + msg239074
2015-03-23 20:42:41paul.mooresetmessages: + msg239062
2015-03-23 20:13:45paul.mooresetmessages: + msg239058
2015-03-23 20:09:10steve.dowersetmessages: + msg239057
2015-03-23 19:38:07paul.mooresetfiles: + addpath.patch
keywords: + patch
messages: + msg239055
2015-03-23 19:32:23paul.mooresetmessages: + msg239054
2015-03-23 19:30:07paul.mooresetmessages: + msg239053
2015-03-23 19:22:46paul.mooresetmessages: + msg239052
2015-03-23 19:21:16paul.mooresetmessages: + msg239051
2015-03-23 19:05:32steve.dowersetmessages: + msg239050
2015-03-23 16:46:14paul.mooresetmessages: + msg239044
2015-03-23 16:40:12steve.dowersetnosy: + paul.moore
messages: + msg239043
2015-03-23 01:06:32BreamoreBoysetmessages: + msg238966
2014-07-30 15:46:16BreamoreBoysetnosy: + BreamoreBoy, zach.ware, steve.dower
messages: + msg224318
2012-10-26 15:28:20Emil.Styrkesetmessages: + msg173853
2012-10-26 13:26:38tim.goldensetnosy: + tim.golden
messages: + msg173842
2012-10-26 13:25:54tim.goldensetnosy: + christian.heimes
2012-10-26 12:41:22Emil.Styrkecreate