New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IDLE: Have the shell mimic terminal handling of \r and \b control characters in outputs #82008
Comments
IDLE's shell doesn't currently handle \r and \b in any special way; they are written the the Tk Text widget which displays them in strange, system-dependent ways. These are often used to show continuously updated progress, e.g. in text-based progress bars, without flooding the output, since they allow overwriting previously written output. If we implement handling for \r and \b, progress indicators such as these could finally work properly in IDLE's shell. To make things worse, Tk's Text widget becomes increasingly slow when it wraps very long lines. IDLE's shell must wrap lines, and is therefore prone to such slowdowns. Attempting to show updating progress info using \r or \b results in such increasingly long lines of output, eventually slowing IDLE's shell down to a crawl. (The recent addition of squeezing of long outputs help for the case of single, very long outputs, but not with many short strings written on a single line.) As a recent example, the basic Tensorflow tutorial shows such progress information for several of its stages. Due to the lack of handling for these control characters, it is practically unusable in the IDLE shell due to this. See issue bpo-37762 about this. Since the shell aims to closely emulate using an interactive terminal session, and since showing progress is so common in interactive work, I propose to add special handling of these two control characters in outputs written to the shell window. Related issues: bpo-23220: Documents input/output effects of how IDLE runs user code (originally titled "IDLE does not display \b backspace correctly") bpo-24572: IDLE Text Output With ASCII Control Codes Not Working (marked as duplicate of bpo-23220) bpo-37762: IDLE very slow due a super long line output in chunks StackOverflow: what is the difference between cmd and idle when using tqdm? (https://stackoverflow.com/questions/35895864/what-is-the-difference-between-cmd-and-idle-when-using-tqdm) |
See PR #59416 with a working implementation. |
I agree -- this was added to Emacs a long time ago and it makes a big difference for people (like myself) who do a lot of work in Emacs shell windows. I imagine it's the same for IDLE. |
In this post, I slightly expand the terminal mode proposal and define the context in which I will accept it. The context will be the subject of other issues. In the next post (tomorrow) I will try to define a reviewable specification for the terminal mode. The proposal that IDLE's should *always* respond to \b and \r "properly I noted in bpo-23220 that one solution for people who wanted terminal-like behavior would be to add an option to execute code being edited to the system terminal. However, this a) throws away the benefits of running in Shell and b) would not apply to interactive exploration. While running in a terminal is needed for certain use cases, it is not needed for handling control of line output. What Shell should do with control characters in output depends on the users goal. I propose that Shell should have 3 display modes that treat ascii control codes differently for different use cases.
To fulfill the goal of Development mode, more is needed. Astral chars, instead of causing an exception, should be replaced by a string consisting of their \U escape and tagged as a replacement. This is needed anywhere astral chars can appear. It would also be nice to get the codepoint and name of any displayed character. If implemented, this should be added to the base editor class.
|
Terry, thanks for the detailed writing of your thoughts on the matter and their context. Serhiy's argument (in msg246602) is that different terminals interpret different control characters in different ways, and that we have no way of unifying their behavior. This is true, and I will add that we should also not aim to fully emulate any single terminal nor multiple types of terminals. On the other hand, there is a common ground where the vast majority of terminals do, in fact, behave very similarly WRT control characters. For example, IDLE already does interpret '\n' in a special way (partially because the underlying Tk text widget does). The same goes for '\t'. I argue that we should instead *slightly* expand the set of control characters which IDLE interprets, to include a few more which are universally treated in a consistent manner. Looking at the list of ASCII control characters on Wikipedia[1], I don't think any beyond \a, \b, \n, \r and \t are universal enough and in common enough use to merit inclusion. (For reference, \a is for "bell".) ..[1]: https://en.wikipedia.org/wiki/C0_and_C1_control_codes I find the suggestion to have more than a single "mode" for the IDLE shell contrary to the "simple and novice friendly" design principle that we are aiming for. I also think that it would bring little added benefit to the great majority of our users. As for other control characters and astral characters, I very much agree that we could do better than to have them often "garbled" as done by the Tk text widget. I think this should be dealt with as a separate issue. |
Note that most of the issues with special characters described in Terry latest comment have been addressed by the fix for bpo-13153. Can we continue discussing just properly rendering \r and \b, e.g. as done in PR #59416? Let's just decide whether we want this or not. My main point in favor is that we already have special support for two similar control characters, \n and \t. IMO we should expand this to the other very standardized ones, \r and \b, and perhaps \a ("bell") as well. |
As the creator of this issue, I'm reverting the name back to the original, since I did not propose to add a new "mode" to the IDLE shell. Such a proposal could be a separate issue. I'm unfortunately going to mark close this as "rejected". If in the future there is interest in this again, it may be reopened. |
I will highlight the fact that IDLE's shell currently allows no way of showing a progress indicator, beyond extremely simple bar that only fill up from left to right, without any indication of the rightmost limit, a percentage indicator, or a rotating progress indicator. Specifically, all common progress indicator libraries do not work properly in IDLE. This is a true detriment to interactive work. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: