classification
Title: IDLE - deprecate running without a subprocess
Type: enhancement Stage: needs patch
Components: IDLE Versions: Python 3.6
process
Status: open Resolution:
Dependencies: 18823 Superseder:
Assigned To: Nosy List: Ramchandra Apte, Torgil Svensson, amaury.forgeotdarc, asvetlov, ideasman42, kbk, larry, ned.deily, python-dev, rhettinger, roger.serwy, terry.reedy
Priority: normal Keywords: patch

Created on 2012-10-03 22:13 by roger.serwy, last changed 2015-11-23 19:27 by terry.reedy.

Files
File name Uploaded Description Edit
idle_deprecate.patch roger.serwy, 2012-10-03 22:13 review
idle_deprecate_rev1.patch roger.serwy, 2012-10-05 18:47 review
Messages (25)
msg171911 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-03 22:13
Removing the option of running IDLE without a subprocess would simplify the maintenance of IDLE. This should be deprecated for 3.4 and then fully removed in 3.5.

Here's a link to a brief discussion on IDLE-dev: http://mail.python.org/pipermail/idle-dev/2012-June/003124.html

Attached is an initial patch to issue a deprecation message.
msg171948 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2012-10-04 14:04
Make it pop up a dialog with the message too in the patch, please.
msg172060 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-10-05 08:02
A popup menu on every invocation of IDLE would be a very user-unfriendly thing to do.  If it's possible that a print to stderr might not be visible to a user, another solution might be to use the approach in Lib/idlelib/macosxSupport.py tkVersionWarning which causes the warning message to show up in the PyShell window.
msg172109 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-10-05 18:47
I agree that a message within the shell would be more informative to the user. The _rev1 patch also adds a message to the shell.
msg172115 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-10-05 19:07
Looks good for me.
msg172117 - (view) Author: Roundup Robot (python-dev) Date: 2012-10-05 19:17
New changeset 0430986a8c03 by Andrew Svetlov in branch 'default':
Issue #16123: IDLE - deprecate running without a subprocess.
http://hg.python.org/cpython/rev/0430986a8c03
msg172119 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-10-05 19:18
Change priority to deferred blocker for reminding to remove no-subprocess mode in 3.5.

Thanks, Roger.
msg176954 - (view) Author: Campbell Barton (ideasman42) Date: 2012-12-04 20:34
Hi, I just found a use for this feature, with `use_subprocess` being optional its possible to load idle into a running python session.

--
    import idlelib
    import idlelib.PyShell
    idlelib.PyShell.main()
--

This allows you to load idle inside an application that embeds python, is there some other way to do this without this option?
msg176967 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2012-12-05 02:29
Campbell, your example runs IDLE *with* a subprocess. The "use_subprocess" flag defaults to True.
msg176968 - (view) Author: Campbell Barton (ideasman42) Date: 2012-12-05 06:31
@Roger, I should have mentioned I had to add "-n" to the argv so subprocess would be disabled.
msg176972 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-12-05 08:29
I think that usage is not officially supported, so we cannot grant this way will work forever.
msg182629 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-02-22 04:30
FWIW, I help a lot of people with IDLE problems and sometimes the -n option is the only thing that saves them.  ISTM that removing it is not such as a good idea.
msg182630 - (view) Author: Roger Serwy (roger.serwy) * (Python committer) Date: 2013-02-22 04:47
@Raymond: What is so broken about IDLE that requires using "-n" to fix it? Let's try to fix those problems rather than keeping up the maintenance burden of supporting two execution modes.
msg182676 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-02-22 17:26
The issue is usually with firewalls, security software, socket issues, etc.  While the root problem lies outside IDLE, often the simplest way to get someone up and running is to use the -n option.

This is one of many annoyances that arise when teaching students Python using IDLE:
* Preferences window crashing
* Default font-size on a retina mac is tiny
* Inability to shutoff the prominent warning messages
* Control-A goes to the beginning of the line, past the >>> prompt.
* On Windows, IDLE sometimes has a two second delay before it runs scripts
* Students find that IDLE mysteriously pastes a clipboard into the interactive prompt unintentionally
* It is a common complaint that IDLE hangs
* Getting the correct Tcl/Tk setup on Macs is problematic.
* Starting IDLE from the command line emits messages about "setCanCycle is deprecated"
msg182711 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-02-23 03:39
On 22 February 2013 22:56, Raymond Hettinger <report@bugs.python.org> wrote:

>
> Raymond Hettinger added the comment:
>
> The issue is usually with firewalls, security software, socket issues,
> etc.  While the root problem lies outside IDLE, often the simplest way to
> get someone up and running is to use the -n option.
>
> This is one of many annoyances that arise when teaching students Python
> using IDLE:
> * Preferences window crashing
> * Default font-size on a retina mac is tiny
> * Inability to shutoff the prominent warning messages
> * Control-A goes to the beginning of the line, past the >>> prompt.
> * On Windows, IDLE sometimes has a two second delay before it runs scripts
> * Students find that IDLE mysteriously pastes a clipboard into the
> interactive prompt unintentionally
> * It is a common complaint that IDLE hangs
> * Getting the correct Tcl/Tk setup on Macs is problematic.
> * Starting IDLE from the command line emits messages about "setCanCycle is
> deprecated"
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue16123>
> _______________________________________
>

Not experienced any of the problems on Linux.
msg182712 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-02-23 03:46
Roger has submitted patches for some of the items on your list (off topic for this issue), especially crashes and stalls. Some still need review and commits. Mac tends to be the most difficult platform to get tests on.

While these would be better discussed on other issues or on idle-sig (mirrored on gmane), I will briefly comment on a few.

Cntl-A selects the entire window on Windows, while Home sends the cursor to the beginning of the line. On the current command line, it first goes after the prompt, and alternates before and after with repetition. This is kbk's design. On old command lines, it only goes before, and that is a bug to be fixed. I think there is an issue, but cannot find it.

I have never seen unprovoked pasting, nor an unexpected startup delay when running from the editor.

I cannot find 'setCanCycle' in 2.7 or 3.3 Lib/idlelib/*.py and 'deprecated' only appears once, in a comment. I expanded the 'setCanCycle' to the entire cpython repository and still did not find it, so that is a complete mystery.
msg184995 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2013-03-22 17:58
> The issue is usually with firewalls, security software, socket issues, etc

Surely nowadays there are better ways than sockets to communicate between processes? Pipes?
msg196024 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-23 20:41
Slight correction: While I do not believe I have seen *clipboard* contents pasted, after running a program from the editor that prints (to the shell), I *have* seen the last line reprinted when trying to enter something new as the prompt. This sometimes might look like pasting from the clipboard. I sometimes have to hit enter a couple of times and ignore the SyntaxError to get a clean prompt. I have no idea if this is exclusive to running in a subprocess; it should be fixed independently of this issue.

The idea of using pipes is being explored on Idle-sig.
msg196057 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-23 23:37
I opened #18823 "Idle: use pipes instead of sockets to talk with user subprocess" I think this or a socket fix is needed before we remove -n.
msg213716 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-03-16 05:17
If the 3.4 changes are done, can we either close this or remove 3.4 from the versions?
msg213727 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-16 14:42
Every enhancement issue can be bumped to 3.5.
msg222957 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-07-13 17:40
It would be nice if we are able to to this in 3.5, because the dependecy is accomplished, but not doing so is not a release blocker.
msg249045 - (view) Author: Torgil Svensson (Torgil Svensson) Date: 2015-08-24 11:46
I frequently use IDLE as an interactive shell to Python with the Editor attached. It's very nice for this purpose as it's lightweight and embedded in Python delivery. This is only possible with -n flag as without it each press on F5 will restart the interpreter and I lose all variables, graphs etc.

Under 2.7 (not related to this thread) I'm also unable to use interactive graphs [ ion()] in matplotlib without the -n flag.

Is there another way to run the module without restarting interpreter without the -n option?

Will 2.7 be affected by this change in the future?
msg249067 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-08-24 17:04
Torgil, thank you for the interesting feedback. I will respond to your four paragraphs slightly out of order.

1. Starting Idle with '-n', I verified that running 'print(a)' from the editor works after entering 'a=1' in the shell. The intended purpose of the editor is to edit files that run as is.  From this viewpoint, the carryover behavior is a bug.  Idle's original single process mode, now accessed with -n, only remains because communicating with a subprocess via sockets turns out to be a bit unreliable.

3. Using an editor as a extension of the shell is an alternate use.  It does not have to be tied to -n.  I have been thinking about alternate run modes and added 'run as input' to my list.  A useful version might be this: don't require a name for the editor buffer; don't restart; enter each statement at a prompt and run as if typed.  The latter would make all the code run in Shell visible and add each statement to the history list.

2. I am a bit puzzled by your matlab comment. When tkinter code is run from editor in a separate process, it successfully grabs keyboard and mouse input.  (One of the reasons for the shift from 1 to 2 processes was to disentangle user tkinter guis from the Idle tkinter gui.) On the other hand, running
  import msvcrt; k = msvcrt.kbhit()
on Windows does not work with either 1 or 2 processes because the Idle gui retains input focus.

If you want to pursue this, please open a new issue on the tracker or post to the idle-sig list or its news.gmane.org mirror gmane.comp.python.idle. In either case, say more about what 'not work' means.

4.This issue was opened before PEP 434 and before 2.7 maintenance was extended from 5 years (expiring now as 3.5 is released) to 10 years.  Hence the deprecation notice was not added to 2.7. I do not expect this to change.
msg255213 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-11-23 19:27
I have some different ideas for this issue that would not necessarily involve removing -n.  While -n is can be an inferior experience for users (user code blocks the GUI, other interference, especially tkinter), the reason given for deprecation is to simplify IDLE maintenance by eliminating alternate code paths.

Summary: a new rpc_local module might allow us to simplify code now, without removing -n, and provide a path to switching subprocess communication from sockets to pipes.

1. Simplify maintenance by instead isolating -n code in a new 'rpc_local' module that is imported 'as rpc' when -n is given.  (A 'run_local' module might also be needed.)  The goal would be to have the rest of IDLE as oblivious as possible as to where user code is executed.  Other modules would send messages via rpc and not know whether they go to another process (currently via a socket) or or back to the main thread in the same process.  I don't know how far this is possible.  It would certainly involve some refactoring.

2. Make rpc_local less of a dummy by executing user code in a separate thread connected by a pair of Queues (and call it rpc_thread).  I believe this would solve the problem of user code freezing IDLE.  On the other hand, it might make user code importing tkinter worse.  If so, I would consider declaring that unsupported and not run code containing 'tkinter'.  Perhaps I am overlooking some important reason a thread is not already used.

3. If rpc_thread worked, multiprocessing could be used to turn it into a new rpc remote process module that communicated with pipes instead of socket ports.
History
Date User Action Args
2015-11-23 19:27:47terry.reedysetmessages: + msg255213
2015-08-24 17:04:49terry.reedysetmessages: + msg249067
versions: + Python 3.6, - Python 3.5
2015-08-24 11:46:32Torgil Svenssonsetnosy: + Torgil Svensson
messages: + msg249045
2014-07-13 17:40:31terry.reedysetpriority: deferred blocker -> normal

dependencies: + Idle: use pipes instead of sockets to talk with user subprocess
messages: + msg222957
stage: needs patch
2014-03-16 14:42:42terry.reedysetmessages: + msg213727
versions: - Python 3.4
2014-03-16 05:17:42larrysetnosy: + larry
messages: + msg213716
2014-02-02 05:44:46kbksetnosy: + kbk
2013-08-23 23:37:42terry.reedysetmessages: + msg196057
2013-08-23 20:41:19terry.reedysetmessages: + msg196024
2013-05-14 03:42:55roger.serwylinkissue13657 dependencies
2013-03-22 17:58:42amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg184995
2013-02-23 03:46:18terry.reedysetmessages: + msg182712
2013-02-23 03:39:48Ramchandra Aptesetmessages: + msg182711
2013-02-22 17:26:07rhettingersetmessages: + msg182676
2013-02-22 04:47:22roger.serwysetmessages: + msg182630
2013-02-22 04:30:05rhettingersetnosy: + rhettinger
messages: + msg182629
2013-02-22 03:32:22terry.reedysetcomponents: + IDLE
2012-12-05 08:29:09asvetlovsetmessages: + msg176972
2012-12-05 06:31:55ideasman42setmessages: + msg176968
2012-12-05 02:29:27roger.serwysetmessages: + msg176967
2012-12-04 20:34:10ideasman42setnosy: + ideasman42
messages: + msg176954
2012-10-05 19:18:43asvetlovsetpriority: normal -> deferred blocker

messages: + msg172119
2012-10-05 19:17:05python-devsetnosy: + python-dev
messages: + msg172117
2012-10-05 19:07:53asvetlovsetmessages: + msg172115
2012-10-05 18:47:40roger.serwysetfiles: + idle_deprecate_rev1.patch

messages: + msg172109
2012-10-05 16:20:37asvetlovsetnosy: + asvetlov
2012-10-05 08:02:04ned.deilysetmessages: + msg172060
2012-10-04 14:04:23Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg171948
2012-10-03 22:13:49roger.serwycreate