classification
Title: Improve IDLE's "subprocess didn't make connection" message
Type: enhancement Stage: needs patch
Components: Documentation, IDLE Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: docs@python, kbk, lac, markroseman, roger.serwy, terry.reedy, willingc
Priority: normal Keywords: easy

Created on 2015-10-29 20:47 by markroseman, last changed 2017-06-14 02:13 by terry.reedy.

Pull Requests
URL Status Linked Edit
PR 2177 merged terry.reedy, 2017-06-14 00:27
PR 2178 merged terry.reedy, 2017-06-14 01:38
Messages (16)
msg253693 - (view) Author: Mark Roseman (markroseman) * Date: 2015-10-29 20:47
When we create e.g. string.py that shadows a stdlib module needed by IDLE, it would be nice if a better error message could be shown, pointing to that cause.

Original message:

lac at smartwheels:~/junk$ echo "print ('hello there')" >string.py
lac at smartwheels:~/junk$ idle-python3.5
hello there
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.5/idlelib/run.py", line 10, in <module>
    from idlelib import CallTips
  File "/usr/lib/python3.5/idlelib/CallTips.py", line 16, in <module>
    from idlelib.HyperParser import HyperParser
  File "/usr/lib/python3.5/idlelib/HyperParser.py", line 14, in <module>
    _ASCII_ID_CHARS = frozenset(string.ascii_letters + string.digits + "_")
AttributeError: module 'string' has no attribute 'ascii_letters'

IDLE then produces a popup that says:

IDLE's subprocess didn't make connection.  Either IDLE can't stat a subprocess por personal firewall software is blocking the connection. <ok>

--------

I think that life would be a whole lot easier for people if instead we got
a message:

Warning: local file /u/lac/junk/string.py shadows module named string in the 
Standard Library

I think that it is python exec that would have to do this -- though of
course the popup could also warn about shadowing in general, instead of
sending people on wild goose chases over their firewalls.

----------------------

Laura, I think what you want should actually be more-or-less doable in IDLE.

The main routine that starts IDLE should be able to detect if it starts correctly (something unlikely to happen if a significant stdlib module is shadowed), watch for an attribute error of that form and try to determine if shadowing is the cause, and if so, reissue a saner error message.

The subprocess/firewall error is occurring because the ‘string’ problem in your example isn’t being hit right away so a few startup things already are happening. The point where we’re showing that error (as a result of a timeout) should be able to check as per the above that IDLE was able to start alright, and if not, change or ignore the timeout error.

There’ll probably be some cases (depending on exactly what gets shadowed) that may be difficult to get to work, but it should be able to handle most things.


See full thread starting at https://mail.python.org/pipermail/python-dev/2015-October/142061.html
msg253694 - (view) Author: Laura Creighton (lac) Date: 2015-10-29 21:31
Note that the full path name of the file that is doing the shadow
is of important interest to teachers.  We get machines will all sorts of garbage written on them -- the amount of hell caused by a misnamed turtle.py is hard to measure -- and anything as helps in the finding of the file as needs removing or renaming will be eternally welcome.
msg253695 - (view) Author: Laura Creighton (lac) Date: 2015-10-29 21:36
s/machines will/machines with/

(I was tired.)
msg253704 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-10-30 01:01
Laura, I did not understand from your python-list post that you were complaining about shadowing disabling IDLE itself, as opposed to user code.  That is partly because you followed up on Peter Otten complaining because IDLE tries to run user code the same way that python does.

The solution for IDLE itself is for IDLE to do what python does on startup, which is to ensure that the stdlib modules it needs are indeed imported from the stdlib.  I presume that python only prepends '' to sys.path *after* it does its imports.  Example:

>>> import sys; list(sys.modules)
[..., abc, ...]
C:\Users\Terry>echo "print('abc')" >abc  # verified with dir
C:\Users\Terry>python
>>> >>> import sys; sys.modules['abc'].__file__
'C:\\Programs\\Python35\\lib\\abc.py'

So IDLE should remove sys.path[0] from sys.path and, in the user process, restore it only after normal imports. Ignore -n single process mode for now.

Aside from this, the popup message needs to be changed, but better that it occur less often (by the fix above) or never (by switching to pipes?).

As for user code: I have in mind that there should be a doc how-to about tracebacks and exception messages, with a sections giving possible explanations and remedies for specific messages, such as the above.  This I would make messages like the above a link to the how-to entry.  In the meanwhile, we could consider a special case search of error message lines.

-I just noticed that this issue is the result of a pydev thread. Off to read that.
msg253705 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-10-30 01:17
"I think that removing '' from sys.path, so that IDLE can run, is better than a nicer warning that it cannot run. This, of course, requires that sys not be shadowed, so that sys.path can be accessed."  So after importing sys, we should check sys.__file__, or do the manipulation in try: except: <post error message>.  This will be part of a much more careful startup sequence.
msg253706 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-10-30 01:43
Either tkinter and tkinter.messagebox or subprocess must be imported to display a message when IDLE is started from an icon, and there is no console.  With subprocess, '''python -i -c "print('Cannot start IDLE (or whatever)')"''' should work.  If nothing else, this would be a way to display a message when import tkinter fails.
msg253745 - (view) Author: Laura Creighton (lac) Date: 2015-10-30 16:34
Re: misunderstanding

Sorry Terry.  From my end the great 4 day "weekend of mail.python.org needs to be rebuilt from scratch" happened precisely as I was about
to reply to your post.  The reply to Peter Otten showed up later 
as part of my mailer agent trying to send the thing every 4 hours for
a week.  So confusion reigns.
msg253833 - (view) Author: Laura Creighton (lac) Date: 2015-11-01 08:00
can I send a file
msg255578 - (view) Author: Laura Creighton (lac) Date: 2015-11-29 15:47
webmaster got another one of these today.
msg258498 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-01-18 05:14
I have seen another report that is likely due to masking.  I don't know why this has suddenly become seemingly much more common, but I have made fixing the problem a priority.  See new issue #26143.

I would like to narrow this issue to the user process, and broaden it to reporting all currently plausible causes of non-connection.  I revised the title to say this.

I am looking  at https://stackoverflow.com/questions/15888186/cant-run-python-via-idle-from-explorer-2013-idles-subprocess-didnt-make-c to see what causes people have found or think they have found.

1. Antivirus/firewall (4 people, linux iptables, Windows firewall, antivisus , so not a myth ;-)
2. Shadow files, as described above, and in the python directory (about as common, tkinter,py, subprocess.py).
3. Registry problem (some people can fix, but most should re-install after complete removal).
4. Zombie pythonw process (kill with Windows Task manager).
5. Permissions (run as admin, though should not be needed with proper install)
?. Multiple installed versions clash due to misconfiguration?

https://stackoverflow.com/questions/874757/python-idle-subprocess-error
Add:
?. example program that reads and writes (test)
6. Running after stopping run with ^C (temporary, I believe Restart Shell will fix)  Call this a timing issue.
<<Note: I have experienced temporary connection failures myself, but have not written what I did>>
7. Modem rebot (rather obsolete answer for most of us).

There are plenty of miscellaneous possibilities to add even if masking, the most common reason, is fixed.
msg295369 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-07 21:05
#26413 is about fixing shadowing problem.

Current thought for this issue: add a section to IDLE doc listing known possible causes (message above).  Reference section in message, giving web link.
msg295937 - (view) Author: Carol Willing (willingc) * (Python committer) Date: 2017-06-13 16:50
Based on Terry's latest message and recommendation, I've marked this as a Documentation issue and suitable for new contributors.
msg295950 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-13 18:56
Thanks for the reminder to get busy on this ;-). My short notes are insufficient for anyone else to write this.
msg295972 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-14 00:37
I tested (and tweaked) the visible box with this:
from idlelib.run import show_socket_error
show_socket_error(OSError(61, 'connection refused'), ('port', 8888))
URL was tested with cut-and-paste from run.py.

This should be turned into an htest (human view test).  I did not do so now because I wanted get this into 3.6.2, and there is no indication of when the grace-period delay will end.

Message boxes do not allow copy to clipboard.  A nice refinement would be a [View URL] button.  I believe the code to do so is somewhere in IDLE.  Or the F1 Python Docs code, which opens the Windows help version, should be adjustable to open to a specific topic.
msg295975 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-14 01:32
New changeset 188aedf8bb623d41302e10503268b0852ea91134 by terryjreedy in branch 'master':
bpo-25514: Improve IDLE's connection refused message (#2177)
https://github.com/python/cpython/commit/188aedf8bb623d41302e10503268b0852ea91134
msg295977 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-06-14 02:13
New changeset 22d909f8c24bd7768df1a5bf0e52c597ea947cfb by terryjreedy in branch '3.6':
[3.6]bpo-25514: Improve IDLE's connection refused message (#2177) (#2178)
https://github.com/python/cpython/commit/22d909f8c24bd7768df1a5bf0e52c597ea947cfb
History
Date User Action Args
2017-06-14 02:13:17terry.reedysetmessages: + msg295977
2017-06-14 01:38:40terry.reedysetpull_requests: + pull_request2231
2017-06-14 01:32:19terry.reedysetmessages: + msg295975
2017-06-14 00:37:56terry.reedysetmessages: + msg295972
versions: + Python 3.7, - Python 2.7, Python 3.5
2017-06-14 00:27:03terry.reedysetpull_requests: + pull_request2230
2017-06-13 18:58:51terry.reedysetfiles: - dia2.py
2017-06-13 18:56:58terry.reedysetassignee: docs@python -> terry.reedy
messages: + msg295950
2017-06-13 16:50:23willingcsetnosy: + willingc, docs@python
messages: + msg295937

assignee: docs@python
components: + Documentation
keywords: + easy
2017-06-07 21:05:44terry.reedysetmessages: + msg295369
2016-01-18 05:14:05terry.reedysetmessages: + msg258498
title: better startup error messages in IDLE when stdlib modules shadowed -> Improve IDLE's "subprocess didn't make connection" message
2015-11-29 15:47:08lacsetmessages: + msg255578
2015-11-01 08:00:01lacsetfiles: + dia2.py
type: behavior -> enhancement
messages: + msg253833

versions: - Python 3.4
2015-10-30 16:34:02lacsetmessages: + msg253745
2015-10-30 01:43:51terry.reedysetmessages: + msg253706
2015-10-30 01:17:19terry.reedysetmessages: + msg253705
2015-10-30 01:01:11terry.reedysettype: enhancement -> behavior
stage: needs patch
messages: + msg253704
versions: + Python 3.4
2015-10-29 21:36:17lacsetmessages: + msg253695
2015-10-29 21:31:59lacsetnosy: + lac
messages: + msg253694
2015-10-29 20:47:20markrosemancreate