Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE: revise doc for control chars sent to Shell #80014

Closed
DudeRoast mannequin opened this issue Jan 26, 2019 · 7 comments
Closed

IDLE: revise doc for control chars sent to Shell #80014

DudeRoast mannequin opened this issue Jan 26, 2019 · 7 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@DudeRoast
Copy link
Mannequin

DudeRoast mannequin commented Jan 26, 2019

BPO 35833
Nosy @terryjreedy, @vadmium, @miss-islington
PRs
  • bpo-35833: Revise IDLE doc for control codes sent to Shell. #11799
  • [3.7] bpo-35833: Revise IDLE doc for control codes sent to Shell. (GH-11799) #11800
  • Files
  • mousenow.py: Code for the program
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2019-02-09.04:45:12.168>
    created_at = <Date 2019-01-26.14:52:50.848>
    labels = ['3.8', 'expert-IDLE', 'type-bug', '3.7']
    title = 'IDLE: revise doc for control chars sent to Shell'
    updated_at = <Date 2019-02-09.04:45:12.167>
    user = 'https://bugs.python.org/DudeRoast'

    bugs.python.org fields:

    activity = <Date 2019-02-09.04:45:12.167>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2019-02-09.04:45:12.168>
    closer = 'terry.reedy'
    components = ['IDLE']
    creation = <Date 2019-01-26.14:52:50.848>
    creator = 'Dude Roast'
    dependencies = []
    files = ['48079']
    hgrepos = []
    issue_num = 35833
    keywords = ['patch', 'patch', 'patch']
    message_count = 7.0
    messages = ['334394', '334400', '334401', '334403', '334405', '335124', '335125']
    nosy_count = 4.0
    nosy_names = ['terry.reedy', 'martin.panter', 'miss-islington', 'Dude Roast']
    pr_nums = ['11799', '11800']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue35833'
    versions = ['Python 3.7', 'Python 3.8']

    @DudeRoast
    Copy link
    Mannequin Author

    DudeRoast mannequin commented Jan 26, 2019

    Whenever I try to use Backspace(\b) it always prints square boxes instead of deleting previous string.

    Code Down:-

    import pyautogui
    print("Press Ctrl+c to quit")

    try:
    while True:
    x,y = pyautogui.position();
    positionStr = "X: " + str(x).rjust(4) + " Y: " + str(y).rjust(4)
    print(positionStr,end ='')
    print('\b'*len(positionStr),end='',flush=True)

    except KeyboardInterrupt:
    print("\nDone")

    O/P:-

    Press Ctrl+c to quit
    X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������X: 317 Y: 261���������������
    Done

    @DudeRoast DudeRoast mannequin added the 3.7 (EOL) end of life label Jan 26, 2019
    @DudeRoast DudeRoast mannequin assigned terryjreedy Jan 26, 2019
    @DudeRoast DudeRoast mannequin added topic-IDLE type-bug An unexpected behavior, bug, or error labels Jan 26, 2019
    @vadmium
    Copy link
    Member

    vadmium commented Jan 26, 2019

    I suspect Idle just passes control characters directly to an underlying Text or similar TK widget. As far as I know, TK only documents behaviour for tabs and newlines, not other control characters.

    Last time this was brought up, Terry added a sentence under <https://docs.python.org/3.7/library/idle.html#user-output-in-shell\>, third paragraph, about this:

    “Newline characters cause following text to appear on a new line, but other control characters are either replaced with a box or deleted.”

    @vadmium
    Copy link
    Member

    vadmium commented Jan 26, 2019

    Ment to point to previous bug report: bpo-23220

    @terryjreedy
    Copy link
    Member

    Example code for the tracker should not use 3rd-party modules unless essential. It should be reduced to the minimum testcase needed to show the behavior. In this case, "print('a\b')", resulting, on Windows 10, in "a�", is enough. (On macOS Mohave, the result is just 'a'.) These results are not a bug. The IDLE chapter of the doc, viewable with Help => IDLE Doc, has a section 'User output in Shell', added for 3.6.8 and 3.7.2, that explains. The full relevant text with the key line Martin posted is

    "When a program outputs text, the result is determined by the corresponding output device. When IDLE executes user code, sys.stdout and sys.stderr are connected to the display area of IDLE’s Shell. Some of its features are inherited from the underlying Tk Text widget. Others are programmed additions. Where it matters, Shell is designed for development rather than production runs.
    ...
    Text widgets display a subset of Unicode, the Basic Multilingual Plane (BMP). Which characters get a proper glyph instead of a replacement box depends on the operating system and installed fonts. Newline characters cause following text to appear on a new line, but other control characters are either replaced with a box or deleted. However, repr(), which is used for interactive echo of expression values, replaces control characters, some BMP codepoints, and all non-BMP characters with escape codes before they are output."

    In reviewing this and doing more experiments, I realized that this needs revision. A. tabs jump to the next 8-char tabstop. B. Display replacements for control chars can include spaces and other things. C. Display replacements depend on the operating system (as noted above) and possibly the font. D. Display replacements only occur when the string is displayed, not when stored in the text widget. Selecting and copying a string on the screen copies what is stored in the widget, not what is displayed on the screen. E. Trying to insert non-BMP characters ('\U000nnnnn') in a text widget is an error. (Preventing this even when a user implicitly requests is a separate issue.) F. Printing null and return characters may cause problems. Null (\0, \x00) hands IDLE on Mac. Return (\r, \x0D) is ignored by tk but interpreted and converted to \n when pasted into code.

    I want to add a code block example that illustrate some of these points.

    >>> s = 'abc\t\a<\x02><\r>\b'
    >>> len(s)
    12
    >>> s
    'abc\t\x07<\x02><\r>\x08'  # This is repr(s).
    >>> print(s)  # In IDLE, inserts s into text widget.
    # Visible result varies; try it.

    PS: On hard-copy terminals, where control characters originated, \b only means 'Move the print position back a character.' On 'soft-copy' terminals and consoles, it may or may not also mean 'erase the previous character'.

    @terryjreedy terryjreedy added the 3.8 only security fixes label Jan 26, 2019
    @terryjreedy terryjreedy changed the title Backspace not working IDLE: revise doc for control chars sent to Shell Jan 26, 2019
    @terryjreedy
    Copy link
    Member

    Martin, your suspicion is correct, as verified by selecting and copying the string and pasting into code between quotes. I strongly suspect that tk just sends the string as is to the OS graphics system insert text function, that the latter decides what to display. I ran
    for i in range(1,33): print(f'<{chr(i)}>')
    on Windows console, Windows IDLE, and Mac IDLE (omitting '1' crashes on Mac) and there are multiple inconsistencies. I believe tcl/tk only documents \t and \n because those are the only two control chars whose behavior is consistent across graphics systems. Thanks for digging up the previous issue. I would close as a duplicate if I did not see a need for doc revision.

    @terryjreedy
    Copy link
    Member

    New changeset 8a03ff2 by Terry Jan Reedy in branch 'master':
    bpo-35833: Revise IDLE doc for control codes sent to Shell. (GH-11799)
    8a03ff2

    @miss-islington
    Copy link
    Contributor

    New changeset 3fcfef3 by Miss Islington (bot) in branch '3.7':
    bpo-35833: Revise IDLE doc for control codes sent to Shell. (GH-11799)
    3fcfef3

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants