Title: IDLE Shell: isolate user code input
Type: behavior Stage: needs patch
Components: IDLE Versions: Python 3.9, Python 3.8, Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: cheryl.sabella, rhettinger, taleinat, terry.reedy
Priority: normal Keywords:

Created on 2019-08-20 11:38 by terry.reedy, last changed 2019-11-02 01:03 by terry.reedy.

Messages (3)
msg350000 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-20 11:38
The main operational difference between the standard Python REPL and IDLE's Shell is that the latter operates with complete Python statements rather than physical lines.  Shell keeps the '>>>' prompt, but with the expanded meaning of "Enter a complete Python statement" rather than just "Enter the first line of a Python statement".  It omits the consequently superfluous continuation prompts, which would mostly be a nuisance if left.

Currently, the prompt precedes and indents the first line of code.  This causes multiple problems.  Internally, first lines have to be treated differently from the rest.  This has lead to bugs, mostly but not all fixed.  Externally, indentation depends on the prompt and does not look right or work right, compared to the same code in a proper editor.  And it lead to the use of Tab for indents.

The use of Tab for Shell indents was recognized as a problem by 2005.  #1196946 proposed using the same space indents as in the editor, resulting, for instance, in
>>> if a:
    if b:
KBK rejected this with "Doesn't really solve the problem."

In 2010, #7676 suggested 4 space indents again and added 2 variations:  8 space indents, and 8 followed by 4, etc. OP Cherniavsky Beni noted that Tab indents are "inconsistent with PEP 8; what's worse, it's makes copy-paste code between the shell and editor windows confusing and dangerous!"  Raymond Hettinger later added that tabs are a "major PITA" and "a continual source of frustration for students".

Starting with msg151418 in 1212, my response was much the same as KBK's.  To me, the underlying problem is having the prompt physically indent the first physical line relative to the following lines.  I consider this IDLE's single biggest design wart.  I proposed then 3 possible solutions to avoid the first line non-significant indent.  They are, with current comments, ...

1. Prompt on a line by itself (vertical separation).

This is easy, and allows for an expanded active prompt, such as
  >>> Enter a complete Python statement on the lines below.
This says exactly what the user should do and should help avoid confusion with a command-line prompt.  ("I entered 'cd path' and got 'SyntaxError'".)

Once a statement is entered, the instruction is obsolete. Only '>>>' is needed, to identify input from output. I think putting '>>>' above past input works less well than for current input.  I will come back to this proposal below, after 3.

2. No input prompt; instead mark output (with #, for instance).

Possible, but confronting beginners with no prompt would likely leave them wondering what to do.  But this is a possible savefile format, one that could be run or edited.  (A savefile with only the code would do the same.)

3. Prompt in a margin, as with line numbers (horizontal separation).

In 1214, I realized that the 'margin' should be implemented as a separate sidebar widget, which was initially being developed for editor line numbers.  We now have that, and a shell sidebar should be fairly easy.  I will open a separate issue with a fairly specific design.  Basically, the first lines of input, stderr output, and stdout output would be flagged with, for instance, '>>>', 'err', and 'out'.  This should be done before the additional proposal below. 

IDLE's Shell should isolate user input not only from prompts.  Debug on/off messages, warnings, and delayed program output can also interfere.  I think that IDLE's Shell should consist of an input and output history area with sidebar, fixed prompt and separator line such as in 1. above, and active input area.

The history area, as now, would be read-only except when responding to input() in the running code.  The change is that it would receive all messages from IDLE and asynchronous program output.  The input area would be a specialized editor area.  When input code is run, it would be copied above the separator prompt with '>>>' added to the sidebar, opposite the first line.

I believe that the easiest implementation would be to use a label for the fixed prompt line and a specialized editor that runs statements as entered.  The editing and smart indents would be the same as in a regular editor.  Once this is done, we could discuss refinements such as allowing pasting of multiple statements.
msg350055 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-21 06:47
Adding a Shell sidebar is #37903.
msg355853 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-11-02 01:03
One of the related changes is to add multiple options for saving the shell history.  For editing, one wants code without prompts and output either omitting or commented out.  For doctests, one wants code with '>>> ' and '... ' prompts and unmarked output.

A second would be to *not* print SyntaxError and a new prompt, but to otherwise indicate 'error' and put the cursor after the already highlighted error, as done in the editor.

A third would being able to submit multiline statements for execution with with F5 instead of having to go to the end of the last line and enter a blank line.  This would be especially useful after correcting a syntax error or other typo.
Date User Action Args
2019-11-02 01:03:14terry.reedysetmessages: + msg355853
2019-08-21 06:47:57terry.reedysetmessages: + msg350055
2019-08-20 11:38:43terry.reedycreate