Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shell restart when error message contains non-BMP characters #80879

Closed
TheMathsGod mannequin opened this issue Apr 22, 2019 · 7 comments
Closed

Shell restart when error message contains non-BMP characters #80879

TheMathsGod mannequin opened this issue Apr 22, 2019 · 7 comments
Assignees
@terryjreedy
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@TheMathsGod
Copy link
Mannequin

TheMathsGod mannequin commented Apr 22, 2019

BPO 36698
Nosy @terryjreedy, @serhiy-storchaka, @miss-islington
PRs
  • bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. #16583
  • [3.8] bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583) #16649
  • [3.7] bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583) #16650
    		•
    Superseder
  • bpo-22742: IDLE shows traceback when printing non-BMP character
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2019-10-08.12:03:42.416>
created_at = <Date 2019-04-22.17:15:57.723>
labels = ['3.8', 'expert-IDLE', 'type-bug', '3.7', '3.9']
title = 'Shell restart when error message contains non-BMP characters'
updated_at = <Date 2019-10-08.12:03:42.411>
user = 'https://bugs.python.org/TheMathsGod'

    bugs.python.org fields:

    activity = <Date 2019-10-08.12:03:42.411>
actor = 'serhiy.storchaka'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2019-10-08.12:03:42.416>
closer = 'serhiy.storchaka'
components = ['IDLE']
creation = <Date 2019-04-22.17:15:57.723>
creator = 'TheMathsGod'
dependencies = []
files = []
hgrepos = []
issue_num = 36698
keywords = ['patch']
message_count = 7.0
messages = ['340662', '340677', '353932', '353968', '354187', '354189', '354191']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'serhiy.storchaka', 'miss-islington', 'TheMathsGod']
pr_nums = ['16583', '16649', '16650']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = '22742'
type = 'behavior'
url = 'https://bugs.python.org/issue36698'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    @TheMathsGod
    Copy link
    Mannequin Author

    TheMathsGod mannequin commented Apr 22, 2019

    When attempting to raise an error with a message containing non-BMP characters (Unicode ordinals above U+0xFFFF), the python shell displays a long traceback including several UnicodeEncodeErrors and then restarts. Example:

    >>> raise Exception('\U0001f603')
Traceback (most recent call last):
  File "<pyshell#84>", line 1, in <module>
    raise Exception('\U0001f603')
Traceback (most recent call last):
  File "<pyshell#84>", line 1, in <module>
    raise Exception('\U0001f603')
Traceback (most recent call last):
  File "D:\Python37\lib\idlelib\run.py", line 474, in runcode
    exec(code, self.locals)
  File "<pyshell#84>", line 1, in <module>
Traceback (most recent call last):
  File "D:\Python37\lib\idlelib\run.py", line 474, in runcode
    exec(code, self.locals)
  File "<pyshell#84>", line 1, in <module>
Exception: 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Python37\lib\idlelib\run.py", line 144, in main
    ret = method(*args, **kwargs)
  File "D:\Python37\lib\idlelib\run.py", line 486, in runcode
    print_exception()
  File "D:\Python37\lib\idlelib\run.py", line 234, in print_exception
    print_exc(typ, val, tb)
  File "D:\Python37\lib\idlelib\run.py", line 232, in print_exc
    print(line, end='', file=efile)
  File "D:\Python37\lib\idlelib\run.py", line 362, in write
    return self.shell.write(s, self.tags)
  File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
    value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
  File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
    return self.asyncreturn(seq)
  File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
    return self.decoderesponse(response)
  File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
    raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Python37\lib\idlelib\run.py", line 158, in main
    print_exception()
  File "D:\Python37\lib\idlelib\run.py", line 234, in print_exception
    print_exc(typ, val, tb)
  File "D:\Python37\lib\idlelib\run.py", line 220, in print_exc
    print_exc(type(context), context, context.__traceback__)
  File "D:\Python37\lib\idlelib\run.py", line 232, in print_exc
    print(line, end='', file=efile)
  File "D:\Python37\lib\idlelib\run.py", line 362, in write
    return self.shell.write(s, self.tags)
  File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
    value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
  File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
    return self.asyncreturn(seq)
  File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
    return self.decoderesponse(response)
  File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
    raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\Python37\lib\idlelib\run.py", line 162, in main
    traceback.print_exception(type, value, tb, file=sys.__stderr__)
  File "D:\Python37\lib\traceback.py", line 105, in print_exception
    print(line, file=file, end="")
  File "D:\Python37\lib\idlelib\run.py", line 362, in write
    return self.shell.write(s, self.tags)
  File "D:\Python37\lib\idlelib\rpc.py", line 608, in __call__
    value = self.sockio.remotecall(self.oid, self.name, args, kwargs)
  File "D:\Python37\lib\idlelib\rpc.py", line 220, in remotecall
    return self.asyncreturn(seq)
  File "D:\Python37\lib\idlelib\rpc.py", line 251, in asyncreturn
    return self.decoderesponse(response)
  File "D:\Python37\lib\idlelib\rpc.py", line 271, in decoderesponse
    raise what
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 11-11: Non-BMP character not supported in Tk

    =============================== RESTART: Shell ===============================

    >>

    I presume the error is caused by Tk being unable to display the characters in the error message, but being forced to anyway by the traceback, causing a series of UnicodeEncodeErrors. Perhaps the error handler should use repr() or similar methods to convert the message into a displayable form?

    @TheMathsGod TheMathsGod mannequin added the 3.7 (EOL) end of life label Apr 22, 2019
    @TheMathsGod TheMathsGod mannequin assigned terryjreedy Apr 22, 2019
    @TheMathsGod TheMathsGod mannequin added topic-IDLE type-bug An unexpected behavior, bug, or error labels Apr 22, 2019
    @terryjreedy
    Copy link
    Member

    You are correct about reason. This is essentially a duplicate of bpo-22742, but I am leaving it open for now because the behavior is worse and there is additional information in the tracebacks. The lines for IDLE internals are normally removed, but the exception during the exception prevented that.

    @terryjreedy terryjreedy added the 3.8 only security fixes label Apr 22, 2019
    @serhiy-storchaka
    Copy link
    Member

    Mostly fixed by PR 16545 (see bpo-13153). The original example is now passed.

    >>> raise Exception('\U0001f603')
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    raise Exception('\U0001f603')
Exception: 😃
>>>

    But you can still cause a restart of the shell by using surrogate characters:

    >>> raise Exception('\ud800')
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    raise Exception('\ud800')
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    raise Exception('\ud800')

    ================================ RESTART: Shell ================================

    >>

    @terryjreedy
    Copy link
    Member

    As with bpo-22742, the problem, without PR 16583, only occurs with *nix. On Windows, the traceback ends with "Exception: �" and a prompt is printed.

    @terryjreedy terryjreedy added the 3.9 only security fixes label Oct 4, 2019
    @serhiy-storchaka
    Copy link
    Member

    New changeset b690a27 by Serhiy Storchaka in branch 'master':
    bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
    b690a27

    @miss-islington
    Copy link
    Contributor

    New changeset a1f4500 by Miss Islington (bot) in branch '3.8':
    bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
    a1f4500

    @miss-islington
    Copy link
    Contributor

    New changeset 4f962ec by Miss Islington (bot) in branch '3.7':
    bpo-36698: IDLE no longer fails when write non-encodable characters to stderr. (GH-16583)
    4f962ec

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees

    @terryjreedy terryjreedy

    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @serhiy-storchaka @terryjreedy @miss-islington