Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows #58781

Open
clikkeb mannequin opened this issue Apr 14, 2012 · 16 comments
Open

IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows #58781

clikkeb mannequin opened this issue Apr 14, 2012 · 16 comments
Assignees
Labels
3.10 only security fixes OS-windows topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@clikkeb
Copy link
Mannequin

clikkeb mannequin commented Apr 14, 2012

BPO 14576
Nosy @terryjreedy, @serwy, @bitdancer, @asvetlov, @roseman

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = None
created_at = <Date 2012-04-14.10:41:05.356>
labels = ['expert-IDLE', 'type-bug', '3.10']
title = 'IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows'
updated_at = <Date 2020-06-08.00:35:50.375>
user = 'https://bugs.python.org/clikkeb'

bugs.python.org fields:

activity = <Date 2020-06-08.00:35:50.375>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2012-04-14.10:41:05.356>
creator = 'clikkeb'
dependencies = []
files = []
hgrepos = []
issue_num = 14576
keywords = []
message_count = 16.0
messages = ['158253', '158288', '158367', '158387', '158404', '158432', '158534', '180121', '185892', '212257', '212296', '215588', '228267', '252040', '252041', '252062']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'roger.serwy', 'r.david.murray', 'asvetlov', 'markroseman', 'clikkeb', 'Divyanshu', 'John Gray']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14576'
versions = ['Python 3.10']

@clikkeb
Copy link
Mannequin Author

clikkeb mannequin commented Apr 14, 2012

It's a common issue that IDLE cannot start on Windows because

"IDLE's subprocess didn't make connection.Either IDLE can't start a subprocess or personal firewall software is blocking the connection."

Everyone claim that the user should set the firewal so that IDLE can connect to subprocess via loopback. I found, instead, that this issue is often caused by an incorrect or missing setting of one the following variables: HOME, USERPROFILE or the combination of HOMEPATH and HOMEDRIVE; if these variables don't represent an existent and writable directory, the error occurs.
Try setting HOMEPATH to an unexistent or unwritable directory just before launching idle.bat script.
Check IdleConf.GetUserCfgDir() in module configHandler.py, where the function os.path.expanduser is called to get the user home directory. You might also temporarly patch GetUserCfgDir, setting the userDir variable to an unexistent path just after it has been initialized via os.path.expanduser (line 202).
Should be considered a check on expanduser behaviour?

OS: Windows XP
Python version: 3.2.2

clikkeb

@clikkeb clikkeb mannequin added the topic-IDLE label Apr 14, 2012
@serwy
Copy link
Mannequin

serwy mannequin commented Apr 14, 2012

I can confirm that setting HOMEPATH to a non-existent directory will prevent IDLE from starting when using idle.bat. If you modify idle.bat such that python.exe is called instead of pythonw.exe, then IDLE starts normally, but with this console message:

Warning: os.path.expanduser("~") points to
C:\Users\doesnotexist
but the path does not exist.

Normally, the message is written to sys.stderr, which is None when using pythonw. See bpo-13582 for more information.

@clikkeb
Copy link
Mannequin Author

clikkeb mannequin commented Apr 15, 2012

Thanks for your answer.

Trying to understand how IDLE uses HOMEPATH and USERPROFILE Windows
variables, I have found the following information:

  1. it seems that when executed via Windows command prompt (cmd.exe),
    os.path.expanduser refers to USERPROFILE to determine the user's
    home directory;
  2. analizing the stack traces of the first calls to
    IdleConf.GetUserCfgDir, you can see that GetUserCfgDir (which
    calls expanduser) is called three times during IDLE startup:
    the first two times it refers to USERPROFILE, the third (called
    via run.py, probably after starting a subprocess) it refers to
    the combination of HOMEDRIVE and HOMEPATH. (???)
  3. when you start IDLE using pythonw, sys.stderr.write(warn) seems
    to raise an AttributeError exception, which is unhandled. This
    causes IDLE to stop running when you either start IDLE that way
    and the user's home directory is unreachable.

Due to the Python's tricky behaviour in determining the Windows
user's home directory, my opinion would be to consider if it is
worth to go further with this discussion or if it could produce
a benefit to IDLE. For sure, it gave me a little bit of headache.

clikkeb.

@bitdancer
Copy link
Member

It certainly is worthwhile pursing this in some fashion, since at the very least the existing error message needs to be improved. But perhaps there is something more that can be done to gracefully handle this case, instead. I think the next interesting question is to find out why an invalid home directory causes idle to not start up. There's no obvious reason why that should be the case in principle, so I suspect there's some error handling missing somewhere.

This probably also applies to 2.7; someone should test that.

@bitdancer bitdancer added the type-bug An unexpected behavior, bug, or error label Apr 16, 2012
@clikkeb
Copy link
Mannequin Author

clikkeb mannequin commented Apr 16, 2012

I think that lines 207-210 of GetUserCfgDir should be modified like this:

try:
sys.stderr.write(warn)
except (IOError, AttributeError): # <----
pass #^^^^^^^^^^^^^^

because when you start IDLE via pythonw.exe (that sets sys.stderr to "None"),
the function call sys.stderr.write(warn) raises the following exception:

AttributeError: 'NoneType' object has no attribute 'write'

and IDLE stops running without displaying any exception error, because that
exception is unhandled.

There is a funcion call to sys.stderr.write also at line 222, just before a
"raise SystemExit" statement, which makes ininfluent the missing
AttributeError exception handling.

@asvetlov
Copy link
Contributor

I guess IdleConf should to have flag like 'writable'.
If user environment points to invalid location (or there are no write access) this flag should be set.
It flag can affect IDLE configuration dialog: user should be notified what him changes will not be permanently saved.
Subprocess can check that flag as well if need.

BTW, there are related bpo-8231.

@clikkeb
Copy link
Mannequin Author

clikkeb mannequin commented Apr 17, 2012

It is one of the possible solutions.

In combination with the "writable flag" solution, you might create a class
variable in IdleConf (e.g. "user_cfg") that contains the user's home directory;
such variable will be initialized to an empty string by IdleConf.__init__.
On each call, GetUserCfgDir tries to initialize "user_cfg" only if it is an
empty string, otherwise it returns it's content.

Anyway, keep in mind that in GetUserConfig there's an unhandled exception (AttributeError) that might be the cause of the "IDLE doesn't start..." issue reported by users.

@terryjreedy
Copy link
Member

I think the next interesting question is to find out why an invalid home directory causes idle to not start up. There's no obvious reason why that should be the case in principle,

As Roger showed, IDLE will start up with an invalid home directory.

so I suspect there's some error handling missing somewhere

Yes, the uncaught exception raised when attempting to display a non-essential but user-useful messages (this is the third issue I know of about this). There are 5 places in configHandler.py where a warning is directly printed to sys.stderr. There are two similar writes in PyShell.py. The first warning is the problem here, but any of them could be a problem and all should be handled better.

Lib/idlelib\PyShell.py: 1357: sys.stderr.write("Error: %s\n" % str(msg))
Lib/idlelib\PyShell.py: 1358: sys.stderr.write(usage_msg)
Lib/idlelib\configHandler.py: 209: sys.stderr.write(warn)
Lib/idlelib\configHandler.py: 223: sys.stderr.write(warn)
Lib/idlelib\configHandler.py: 255: sys.stderr.write(warning)
Lib/idlelib\configHandler.py: 367: sys.stderr.write(warning)
Lib/idlelib\configHandler.py: 624: sys.stderr.write(warning)

Why are warning used instead of message boxes? Perhaps easier, perhaps the latter are not available because the tk loop is not running. Possible solutions:

  • Discard the message by just catching AttributeError (clikkeb) -- at each place indicated above. This would be better than nothing, but only as a temporary measure, and there is a better temporary measure below.

  • Rearrange the start up process to make message boxes available, with default configuraton, before the configuration process.

  • Make a list of messages and display them when it must (on exit, if that is possible with atexit) or in message boxes when it can.

  • Solve the more general problem of writing to None by making sys.stdout and sys.stderr not be None, bpo-13582. At minimum, even as a temporary measure, use something that just swallows output. Something like

import io
class DummyOut(io.IOBase):
    def write(self, s):
        return len(s)

dum = DummyOut()
print(dum.write('sixsix'))
# 6

@terryjreedy terryjreedy changed the title IDLE cannot connect to subprocess - New solution IDLE: closes when writing warnings on Windows Jan 17, 2013
@serwy
Copy link
Mannequin

serwy mannequin commented Apr 3, 2013

This issue triggers the problem described in bpo-13582. It points out problems with IDLE's configuration manager w.r.t. determining the home directory. I changed the title so that it reflects that point.

@serwy serwy mannequin changed the title IDLE: closes when writing warnings on Windows IDLE: resolving home directory for configuration uses HOMEDRIVE, HOMEPATH, and USERPROFILE inconsistently on Windows. Apr 3, 2013
@Divyanshu
Copy link
Mannequin

Divyanshu mannequin commented Feb 26, 2014

I found a weird solution for the problem. Exchange the names of python and pythonw in the python33 folder. This makes the IDLE to call both and as a result python opens in both modes simultaneosly, and the subprocess connection error don't shows up.

@terryjreedy
Copy link
Member

I do not understand what you mean by "Exchange the names of python and pythonw in the python33 folder.". In any case, idle.bat cannot run both simultaneously.

Perhaps idle.bat should have an option to start with python instead of pythonw.

@Divyanshu
Copy link
Mannequin

Divyanshu mannequin commented Apr 5, 2014

I found the root cause for the error regarding python in my pc that shows the error message

"IDLE's subprocess didn't make connection.Either IDLE can't start a subprocess or personal firewall software is blocking the connection."

It was caused due to a software known as proxifier which provides fast and somewhat unrestricted network access on proxy networks. It redirected my connection request from pythonw.exe to the port and IP on which the proxy network is established.

The problem hence can be easily sorted by creating new proxification rules in such softwares, if the cause of the problem can be traced to some similar mechanisms used that may intercept and modify the network requests for other programs.

@Divyanshu Divyanshu mannequin changed the title IDLE: resolving home directory for configuration uses HOMEDRIVE, HOMEPATH, and USERPROFILE inconsistently on Windows. IDLE: "IDLE's subprocess didn't make connection.Either IDLE can't start a subprocess or personal firewall software is blocking the connection." Apr 5, 2014
@terryjreedy
Copy link
Member

Divyanshu, please do not hijack by changing the title to a different issue. There are multiple causes for the message, some unknown. Your non-standard system is different from that of clikkeb.

@terryjreedy terryjreedy changed the title IDLE: "IDLE's subprocess didn't make connection.Either IDLE can't start a subprocess or personal firewall software is blocking the connection." IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows Oct 2, 2014
@JohnGray
Copy link
Mannequin

JohnGray mannequin commented Oct 1, 2015

I hit this issue with an "H:" homedrive that is on a network share, and then in Windows is using "offline files" to keep a local copy. .idlerc was not cached so IDLE worked when online/connected to my work network but not when I was offline.

The temporary workaround was to mark .idlerc as "available offline" when connected. I wanted to document this in case others hit this scenario.

I also noticed that there is a .idlerc directory created in my local user directory c:\users\[username\.idlerc. This one has the recent-files.lst in it. So IDLE is creating two copies of .idlerc - one in the environment-variable-defined (and roaming) home directory, and one in the default local user directory. I would suggest that as this bug is investigated, you keep track of both instances of .idlerc.

@JohnGray
Copy link
Mannequin

JohnGray mannequin commented Oct 1, 2015

For the h: drive issue mentioned above: This was on Python 2.7.10.

@terryjreedy
Copy link
Member

John, thank you for the additional information. 2 copies of .idlerc is definitely bad.

@terryjreedy terryjreedy added the 3.7 (EOL) end of life label Jun 30, 2017
@terryjreedy terryjreedy self-assigned this Jun 30, 2017
@terryjreedy terryjreedy added 3.10 only security fixes and removed 3.7 (EOL) end of life labels Jun 8, 2020
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.10 only security fixes OS-windows topic-IDLE type-bug An unexpected behavior, bug, or error
Projects
Status: No status
Development

No branches or pull requests

4 participants