Author terry.reedy
Recipients Dude Roast, terry.reedy
Date 2019-01-26.22:14:55
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1548540896.13.0.0143928262673.issue35833@roundup.psfhosted.org>
In-reply-to
Content
Example code for the tracker should not use 3rd-party modules unless essential.  It should be reduced to the minimum testcase needed to show the behavior.  In this case, "print('a\b')", resulting, on Windows 10, in "a", is enough. (On macOS Mohave, the result is just 'a'.) These results are not a bug. The IDLE chapter of the doc, viewable with Help => IDLE Doc, has a section 'User output in Shell', added for 3.6.8 and 3.7.2, that explains.  The full relevant text with the key line Martin posted is

"When a program outputs text, the result is determined by the corresponding output device.  When IDLE executes user code, sys.stdout and sys.stderr are connected to the display area of IDLE’s Shell.  Some of its features are inherited from the underlying Tk Text widget.  Others are programmed additions.  Where it matters, Shell is designed for development rather than production runs. 
...
Text widgets display a subset of Unicode, the Basic Multilingual Plane (BMP). Which characters get a proper glyph instead of a replacement box depends on the operating system and installed fonts.  Newline characters cause following text to appear on a new line, but other control characters are either replaced with a box or deleted.  However, repr(), which is used for interactive echo of expression values, replaces control characters, some BMP codepoints, and all non-BMP characters with escape codes before they are output."

In reviewing this and doing more experiments, I realized that this needs revision. A. tabs jump to the next 8-char tabstop.  B. Display replacements for control chars can include spaces and other things.  C. Display replacements depend on the operating system (as noted above) and possibly the font. D. Display replacements only occur when the string is displayed, not when stored in the text widget.  Selecting and copying a string on the screen copies what is stored in the widget, not what is displayed on the screen.  E. Trying to insert non-BMP characters ('\U000nnnnn') in a text widget is an error.  (Preventing this even when a user implicitly requests is a separate issue.)  F. Printing null and return characters may cause problems.  Null (\0, \x00) hands IDLE on Mac.  Return (\r, \x0D) is ignored by tk but interpreted and converted to \n when pasted into code.

I want to add a code block example that illustrate some of these points.

>>> s = 'abc\t\a<\x02><\r>\b'
>>> len(s)
12
>>> s
'abc\t\x07<\x02><\r>\x08'  # This is repr(s).
>>> print(s)  # In IDLE, inserts s into text widget.
# Visible result varies; try it.

PS: On hard-copy terminals, where control characters originated, \b only means 'Move the print position back a character.'  On 'soft-copy' terminals and consoles, it may or may not also mean 'erase the previous character'.
History
Date User Action Args
2019-01-26 22:14:57terry.reedysetrecipients: + terry.reedy, Dude Roast
2019-01-26 22:14:56terry.reedysetmessageid: <1548540896.13.0.0143928262673.issue35833@roundup.psfhosted.org>
2019-01-26 22:14:56terry.reedylinkissue35833 messages
2019-01-26 22:14:55terry.reedycreate