classification
Title: IDLE squeezer: improve unsqueezing and autosqueeze default
Type: enhancement Stage: test needed
Components: IDLE Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: terry.reedy Nosy List: taleinat, terry.reedy
Priority: normal Keywords:

Created on 2019-01-30 03:14 by terry.reedy, last changed 2019-08-25 06:00 by taleinat.

Messages (7)
msg334541 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-01-30 03:14
This issue continues #35196, which fixed some bugs (inconsistencies) in auto-squeezing and sped the scan of output strings for possible auto-squeezing.  This issue has two parts:
1. Make unsqueezing faster and easier for the user.  Details discussed below.
2. Reconsider the parameters and protocol for auto-squeezing.
2a. Increase the default setting for the minimum number of number of lines to squeeze.  The best setting depends on the result of part 1.
2b. Other changes?

(Some of the work might be done on separate PRs on issues that are dependencies of this one.)

If users ask for a big blob of text, they presumably want to read at least some of it, usually from the beginning, and possibly scan up and down.  The most common example is output from help(object), defined in the pydoc and _sitebuiltins modules.

Help first computes a single output string, whether 2 lines (list.append), 600 (itertools) or over 20000 (tkinter).  For modules, one most likely want to see the module docstring.  Those for itertools are about 40 and 80 lines respectively and both list the functions/classes in the module.

With standard interactive python on a text terminal/console, help output is run through a pager.  If more than a full screen, output is paused with a prompt.  I consider both 'more' on Windows and 'less' on Mac to be overall unsatisfactory, especially for naive beginners, and hope we can do overall better on IDLE.

On Windows, the prompt is "-- More --", with no hint of what to do.  One can experiment and discover that Enter means 'display one more line' and Space means 'display one more screenful'.  Or one can guess to enter "more /h" at a console command prompt.  Either way, paging a thousand times to get the entire output for tkinter is not practical.

As far as I can tell from the 'more /h' output, there is no 'dump remaining text' command other than 'p <more than remaining # of lines>'.  What is worse, a Windows console only holds the last N lines displayed, where N defaults to 300 and can be increased to at most 9999. So scrolling back is limited.  This is terrible, especially at the default setting.

On Mac Terminal, the pager is 'less', the prompt is ':', Space and Enter do the same, scrolling is only partially possible and weird, P goes to the top, a number N after space goes down N lines, to an 'END' prompt. As near as I could discover, less refuses to exit until one hits 'q', at which point the help text disappears.  This is true even for the 2-line help for list.append.  Terrible.

On IDLE, without squeezer, the entire text is displayed followed by a fresh 'enter code' prompt ('>>>').  However, for multi-screen text, the user immediately sees only the last lines, not the first.  This is bad, But at least there is a possibility of scrolling up and trying to find the beginning of the text, although this may take several seconds.

When squeezer unsqueezes, it makes the first line, not the last line, visible.  For a long enough output string, the easier way to get to the first line to start reading, other than scrolling, is to squeeze and unsqueeze.  (This applies to open_file.read() also.)  Absent anything better, I now consider this the primary justification for auto-squeezing.

The following should make triggering expansion of a squeeze label easier and faster, in terms of user actions, regardless of how the  label came about.

E1. Add 'Expand' at the top of the context menu.  (I sometimes right click instead of double-clicking the squeeze label.)  (And add 'Help' at the bottom, to display an explanation of Squeezer.)

E2. Add a hot key to expand when the text cursor is on the line with the squeeze label.  After typing 'help(xyz)' and getting a squeeze label, I would prefer to hit <up arrow> <something> and perhaps use navigation keys instead of immediately having to grab the mouse.

E3. Stop the false (or at least out-dated) and confusing warning about many normal text lines causing problems.  The 20000 lines of tkinter help is not an issue on either my years-old Windows desktop and slower years-old MacbookAir.  I once printed over half a million lines , about 40 chars as I remember, on Windows IDLE without issue.

Long lines are a different issue.  'a'*10000 is okay on Windows, but not on Mac.  On the latter, after unsqueezing, and scrolling down to the new prompt, trying to scroll back up results in the OS twirly pause icon for a few seconds.  The natureal response of adding more keys presses and mouse clicks trying to get a response probably made the experience worse.  This reminds me of my previous Windows machine a decade ago.  

A line length warning needs data both on the machine and the max lines of the text.  The latter might be gathered fast enough by checking line lengths in an after loop.


Once the text is expanded, it could be more immediately useful.

U1. If the squeeze label is near the bottom of the window, only the top few lines are made visible.  Instead, put the top of the output at the top of the window to make as many lines as possible visible.  This should be easy.

U2. *Perhaps* we should put the text cursor at the beginning of the output, instead of leaving it at the next prompt, so nagivation keys work.  But this has tradeoffs and I think it should be left until some other stuff is done.


Once we improve unsqueezing, we can consider auto-squeeze changes. 

A1. Increase the default max lines before auto-squeeze.  Changing defaults is problematical because users customizations apply to all versions a user runs.  A new default on a new version will erase a matching custom value set on an older version.  So new values should be unlikely as custom values.

Another issue is using the lines number to check line length.
10000 = 125 X 80.  On my Windows machine, I think 123 (not 125!) might be okay.  But definitely not on Mac.

One could say we should not protect people from the consequence of foolish inputs, but letting IDLE freeze is part of what has lead to 'IDLE is junk' comments (on Stackoverflow and pydev that I know of).

A2. Decouple lines from length.  Regardless of what I thought when reviewing squeezer, I think now that this is crucial.  But try to determine an appropriate length by internal checks instead of adding another configuration value.  Or make the coupling non-linear (maxlen = C*log(maxlines)?)  Or see what adding horizontal scroll does.

A3. (Suggestion from others) Print a screenful of an output block and then squeeze.  The downside to me is two-fold: one will likely need to unsqueeze anyway; this prevents or complicates moving text to the clipboard or a text viewer.  (The label item could keep track of how much was displayed, but then resizing is an issue.)

Some other ideas.

I1. Write a pager.  No.

I2. Add other ways to get back to the start of a text block.  Add 'Previous Prompt' or 'Prompt Up' or ??? (and Prompt next/down) to the unsqueezed menu.  Or add Control/Command PageUp/Down for Prev/Next Prompt to the fixed navigation keys.  (or both.) These sequences are unused for my current keysets.  config-keys.def should be checked.  Some people would likely prefer these to autosqueeze for getting to the beginning of a blob.  

I3. Add 'Copy' and 'View' (and 'Help') to the unsqueezed context menu so one can skip the visible label step.
msg334631 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-01-31 15:40
In my opinion, everything here is relatively minor compared to some other current issues with IDLE, e.g. some of those mentioned by Raymond Hettinger in his comments on #35196.

That being said, a few comments:

> E1. Add 'Expand' at the top of the context menu.  (I sometimes right click instead of double-clicking the squeeze label.)  (And add 'Help' at the bottom, to display an explanation of Squeezer.)

I agree. If you create an issue, I'll make a PR.

> E2. Add a hot key to expand when the text cursor is on the line with the squeeze label.

This existed in previous versions of Squeezer, but I removed it to keep things simple. It can easily be added.

> U1. If the squeeze label is near the bottom of the window, only the top few lines are made visible.

Scrolling the screen for the user should usually be avoided. With scrolling being easy these days (scroll wheels, native scrolling in touchpads, touch screens) I vote to leave this up to the user.

> U2. *Perhaps* we should put the text cursor at the beginning of the output, instead of leaving it at the next prompt, so nagivation keys work.

This would be the case when done via keyboard as suggested in E2. I'm strongly against moving the cursor upon un-squeezing text in general.


Regarding A1-A3 and the rest, I fail to see the point, but have nothing constructive to say.
msg349058 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-05 17:11
A4: Autosqueeze long lines output in chunks.  tensorflow.keras appears to do this.  See #37762.  Easy reproducer:

for i in range(10000):
    print('%6s--------------------------------------------' % i, end='')
msg349066 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-05 19:55
Regarding help(), there's actually a comment in IDLE's code by KBK that it should be opened in a reader (this was before TextViewer was added).  IMO that's a much better approach for long help() outputs, and is simple to implement.
msg349072 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-08-05 20:52
I presume you are proposing to wrap the site-added help with a function that would check object outputs and put them either in the shell unsqueezed or directly into a viewer.  Seems plausible.
msg349475 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-12 15:37
See issue #37768 regarding opening of help(object) output in a separate window.
msg350429 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2019-08-25 06:00
See issue #37827 regarding handling of progress output, as written by tensorflow.keras for example.
History
Date User Action Args
2019-08-25 06:00:37taleinatsetmessages: + msg350429
versions: + Python 3.9
2019-08-12 15:37:38taleinatsetmessages: + msg349475
2019-08-05 20:52:18terry.reedysetmessages: + msg349072
2019-08-05 19:55:28taleinatsetmessages: + msg349066
2019-08-05 17:11:53terry.reedysetmessages: + msg349058
2019-01-31 15:40:05taleinatsetmessages: + msg334631
2019-01-30 03:14:30terry.reedycreate