I wrote a little python script that prints a large 
dictionary to stdout (simply using 'print 
mydictionary').

In fact, the type is irrelevant, what matters is that 
the resulting output had approx. 200,000 characters.

The shell prints the dictionary into a single line, 
which causes the window to be almost non-responding, 
e.g. when I try to scroll the window.

Even on a high-end PC it takes a minute or even 
longer to react to anything.

I use Python 2.4.2 on Windows XP SP2.

I am aware that it is not exactly wise to print such 
large objects, but I usually print return values to 
stdout when I debug a script, and I do not always 
expect an object to be that large.

The average text editor handles such long lines much 
better.

A quick workaround might be to break very long lines 
automagically (perhaps at around column 1000).

PS: I already observed the bug some years ago. I 
think I even submitted it to python or idlefork a 
long time ago but I was unable to find it in the 
buglist.

Logged In: YES 
user_id=593130

I verified this with print 100000*'a', also XP (home) sp2.
The sluggishness continued after getting the prompt back 
and trying to do something simple, like 2+2, taking maybe 
1/2 minute to print 4 and then the >>> prompt again.
The sluggishness *also* continued after restarting the 
shell (^F6).  This indicates that the problem is with the 
window, not with IDLE.  Hope someone can try same on *nix 
system to see if general with TKinter or specific to Win 
systems.

Logged In: YES 
user_id=341410

Generally speaking, most wrapping text controls have issues
with wrapping long lines.  It would seem reasonable to get
the width of the text control in characters, and manually
wrap all incoming lines regardless.  If the existance or not
of real line endings are important, one could mark which
lines are manually wrapped and remove the line endings on
copy (edit->copy, ctrl+c, etc.).

Logged In: YES 
user_id=149084

This is a known problem with Tk/Tcl/tkinter - large output 
scrolls slowly. It's not something that can be fixed in IDLE.  I 
tried it on Arch Linux - IDLE 2.5a0 - Tk 8.4.  250,000 character 
output not too bad , 25 sec, but 10,000 lines of 25 char takes 
over twice that long, so breaking the lines doesn't help.

I don't see any response problem once the output completes.  The 
situation is exponentially worse at 500,000 char.

What is your use case?  IDLE is designed to be an IDE.  Why 
output such massive data?

You may be interested in Squeezer, a Noam Raphael extension to 
IDLEfork.

http://sourceforge.net/tracker/index.php?
func=detail&aid=704316&group_id=9579&atid=309579

I haven't tried it myself, but it might be what you're looking 
for.

Logged In: YES 
user_id=865975

Hi kbk,

well, my use case is debugging. I write a script and run it
with IDLE. It doesn't behave as expected, so I insert a
print statement. Next time I run it, IDLE hangs. Oops, it
was a long array; it should have been an int. Line too long.
Duh.

OK, I wait through it, correct the bug, run it again. What
happens? IDLE hangs again, trying to scroll a long line (of
the previous run).

Never mind, I can always kill the process... Dammit, I
can't! It eats 100% CPU; task manager doesn't respond.

IMHO his takes away one of python's strengths, which is
making quick hacks really quick.

Would you suggest redirecting this issue to tkinter? You
don't seem to consider this an issue at all.

I will give squeezer a try. Or maybe PyDev?

Logged In: YES 
user_id=341410

You can close the window which includes the Shell that has
the huge output, or even reduce the priority of your Idle
shell (you can make it automatic by mucking about with the
shortcut; see the 'start' command).

Logged In: YES 
user_id=149084

It's not that I don't consider it an issue, but I can't
do anything to improve the performance of the Tk text
widget.  IDLE is pure Python.

One thing that comes to mind is to set a maximum line 
length.  If the line exceeds that, print line(:(MAX -100)
+ '...' + line(:-100)  instead of printing the whole thing
which no one wants to look at anyway.

Another thing I've wanted to do is provide the ability
to clear the shell window when it gets too full, w/o
restarting IDLE.

Yes, you might ask the tkinter guys on their mail list,
I'd be interested in hearing their reply.

Logged In: YES 
user_id=1330769

The Squeezer extension works like a charm! It's also been
thoroughly tested by tens of users over the past several years.

Why not include it as one of the default extensions, and
have it enabled by default?

BTW I have a tweaked version of Squeezer (I fixed the line
counting code), if you're interested.

Logged In: YES 
user_id=149084

Sure, please open a patch and supply a diff against
Noam's version.

Logged In: YES 
user_id=1330769

Done.

Logged In: YES 
user_id=149084

Patch 1529353 by Tal Einat

Patch review in issue 1529353.

print(100000*'a') is still a problem in 3.3.

In addition to squeezing, it would be nice (and easy) to add a menu option (and hotkey) to clear the text pane.

@Raymond, see issue6143 for an (outdated) extension to clear the shell window with a hotkey.

The only reason that the IDLE shell is slow is due to the shell's text widget being configured to have wrap="char". If we manually wrapped the output then the shell responds very quickly to rendering really long strings.

The attached proof-of-concept patch (against 2.7 tip) implements manual wrapping. You can type "print('a' * 10**6)" and the shell responds almost instantly when using no-subprocess mode. (The RPC overhead becomes readily apparent when using a subprocess, introducing a large uninteruptable delay. That's another issue.)

I left text wrapping enabled in the shell since the user may be using a variable-spaced font. A possible compromise would be to increase the wrap_index to a large number, like 32768, before IDLE inserts a '\n' into the output. This would mimic the wrapping behavior of the original shell, but keep the shell responsive when you write a very long string to the output.

manual_wrap.patch patches OutputWindow (2.7) to arbitrarily and blindly wrap at 80 chars.  OutputWindow is used directly for grep output, as well as the base for PyShell.  The grep output format is currently "'{}: {}: {}'.format(filepath, linenum, codeline)".  Output lines are typically (for me) > 80 chars and I would not want a fixed-column wrap.  With a right click, one can go to the file and line and this should not be disabled.  Ditto for tracebacks, where code lines are pre-wrapped (with an added indent) onto a second physical line. Wrap at 80 would wrap lines that were originally 80 before having the traceback indent added.  Autowrap should only be applied to user stdout sent to Shell.

Perhaps wrapping should have a window (80-100?) within which we look for a space.  I have about concluded that we should add horizontal scrollbars anyway, since Python and IDLE output lines longer than 80 chars.

To evaluate the patch further, I want to look at how the socket stream is being read.

As long as we are modifying user output before inserting into the text widget, astral chars should be expanded into their unicode escapes. (There are multiple issue for astral chars.  Tk 8.7 reportedly will handle them.) The replacement text should be tagged and colored as such.  Wrapping should not break replacements.  The same could be done for control chars to make them visible.  (Astral char handling is needed for paths also!).

Besides warping text, there has a performance issue inside the RPCServer and Client.

The (console, write, (text, file), {}) command is sent by server `asynccall`->`putmessage`. It should be sent by chunk size to client, and render on IDLE shell.

The result is performed as sent by chunk size, but the client will gather all chunk until it receives all data, then render on IDLE shell.

This cause the shell seems like hanging there, and doing nothing (in REPL, it will output the long string to stdout and so on).

We can manually detect this then manully chunk out (console, write, args, kwargs) command's args size, so that it will look like not hanging there.

The attach patch is a PoC about this.

-----

For the text widget performance, I dislike the wrap method, it shouldn't be a limit to the user on IDLE (GUI IDE), even it can be set to 80 or 100 or whatever.

Squeezer was added last summer, and definitely helps, but I still intend to consider other points raised here.