Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash on REPL mode with long text copy and paste #83390

Closed
corona10 opened this issue Jan 4, 2020 · 5 comments
Closed

Crash on REPL mode with long text copy and paste #83390

corona10 opened this issue Jan 4, 2020 · 5 comments
Labels
3.9 only security fixes type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@corona10
Copy link
Member

corona10 commented Jan 4, 2020

BPO 39209
Nosy @vstinner, @methane, @serhiy-storchaka, @asottile, @corona10, @pablogsal, @miss-islington
PRs
  • bpo-39209: Manage correctly multi-line tokens in interactive mode #17860
  • [3.8] bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860) #17867
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-01-06.16:27:13.760>
created_at = <Date 2020-01-04.14:29:32.160>
labels = ['3.9', 'type-crash']
title = 'Crash on REPL mode with long text copy and paste'
updated_at = <Date 2020-01-06.16:27:13.760>
user = 'https://github.com/corona10'

    bugs.python.org fields:

    activity = <Date 2020-01-06.16:27:13.760>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2020-01-06.16:27:13.760>
closer = 'pablogsal'
components = []
creation = <Date 2020-01-04.14:29:32.160>
creator = 'corona10'
dependencies = []
files = []
hgrepos = []
issue_num = 39209
keywords = ['patch']
message_count = 5.0
messages = ['359290', '359414', '359416', '359432', '359435']
nosy_count = 7.0
nosy_names = ['vstinner', 'methane', 'serhiy.storchaka', 'Anthony Sottile', 'corona10', 'pablogsal', 'miss-islington']
pr_nums = ['17860', '17867']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue39209'
versions = ['Python 3.9']
    @corona10
    Copy link
    Member Author

    corona10 commented Jan 4, 2020

    When I copy and paste the pretty long text into REPL shell.
    REPL shell is crash down with segment fault.

    This issue is only reproducible on macOS,
    but Linux REPL doesn't look like normal behavior.

    [origin text]
    <?xml version="1.0" encoding="iso-8859-1"?>
    <test>
    <Users>
    <fun25>
    <limits>
    <total>0KiB</total>
    <kbps>0</kbps>
    <rps>1.3</rps>
    <connections>0</connections>
    </limits>
    <usages>
    <total>16738211KiB</total>
    <kbps>237.15</kbps>
    <rps>1.3</rps>
    <connections>0</connections>
    </usages>
    <time_to_refresh>never</time_to_refresh>
    <limit_exceeded_URL>none</limit_exceeded_URL>
    </fun25>
    </Users>
    </test>

    [macOS]

    Python 3.9.0a2+ (heads/master:7dc72b8d4f, Jan  4 2020, 23:22:45)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = """<?xml version="1.0" encoding="iso-8859-1"?>
... <test>
...     <Users>
...         <fun25>
...             <limits>
...                 <total>0KiB</total>
...                 <kbps>0</kbps>
...                 <rps>1.3</rps>
...                 <connections>0</connections>
...             </limits>
...             <usages>
...                 <total>16738211KiB</total>
...                 <kbps>237.15</kbps>
...                 <rps>1.3</rps>
...                 <connections>0</connections>
...             </usages>
...             <time_to_refresh>never</time_to_refresh>
...             <limit_exceeded_URL>none</limit_exceeded_URL>
...         </fun25>
...     </Users>
... </test>
... """
Assertion failed: ((intptr_t)(int)(a - line_start) == (a - line_start)), function parsetok, file Parser/parsetok.c, line 324.
[1]    13389 abort      ./python.exe

    [linux]

    Python 3.9.0a2+ (heads/master-dirty:7dc72b8, Jan  4 2020, 23:22:11)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = """<?xml version="1.0" encoding="iso-8859-1"?>
<test>
    <Users>
        <fun25>
            <limits>
                <total>0KiB</total>
                <kbps>0</kbps>
                <rps>true</rps>
                <connections>0</connections>
            </limits>
            <usages>
                <total>16738211KiB</total>
                <kbps>237.15</kbps>
                <rps>true</rps>
                <connections>0</connections>
            </usages>
            <time_to_refresh>never</time_to_refresh>
            <limit_exceeded_URL>none</limit_exceeded_URL>
        </fun25>
    </Users>
</test>
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... """
>>> a
'<?xml version="1.0" encoding="iso-8859-1"?>\n<test>\n    <Users>\n        <fun25>\n            <limits>\n                <total>0KiB</total>\n                <kbps>0</kbps>\n                <rps>true</rps>\n                <connections>0</connections>\n            </limits>\n            <usages>\n                <total>16738211KiB</total>\n                <kbps>237.15</kbps>\n                <rps>true</rps>\n                <connections>0</connections>\n            </usages>\n            <time_to_refresh>never</time_to_refresh>\n            <limit_exceeded_URL>none</limit_exceeded_URL>\n        </fun25>\n    </Users>\n</test>\n'
>>>

    @corona10 corona10 added 3.9 only security fixes type-crash A hard crash of the interpreter, possibly with a core dump labels Jan 4, 2020
    @pablogsal
    Copy link
    Member

    I bisected this to this commit:

    995d9b9 is the first bad commit
    commit 995d9b9
    Author: Anthony Sottile <asottile@umich.edu>
    Date: Sat Jan 12 20:05:13 2019 -0800

    bpo-16806: Fix `lineno` and `col_offset` for multi-line string tokens (GH-10021)

    Lib/test/test_ast.py | 19 +++++++++++++
    Lib/test/test_fstring.py | 32 +++++++++-------------
    Lib/test/test_opcodes.py | 2 +-
    Lib/test/test_string_literals.py | 8 +++---
    Misc/ACKS | 1 +
    .../2018-10-20-18-05-58.bpo-16806.zr3A9N.rst | 1 +
    Parser/parsetok.c | 15 ++++++++--
    Parser/tokenizer.c | 7 +++++
    Parser/tokenizer.h | 5 ++++
    Python/ast.c | 10 +++++--
    Python/importlib.h | 16 +++++------
    Python/importlib_external.h | 18 ++++++------
    Python/importlib_zipimport.h | 8 +++---
    13 files changed, 91 insertions(+), 51 deletions(-)
    create mode 100644 Misc/NEWS.d/next/Core and Builtins/2018-10-20-18-05-58.bpo-16806.zr3A9N.rst

    @corona10
    Copy link
    Member Author

    corona10 commented Jan 6, 2020

    @pablogsal

    Works correct with PR 17860 :)

    Python 3.9.0a2+ (heads/pr/17860:958541d67c, Jan  6 2020, 20:45:49)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = """<?xml version="1.0" encoding="iso-8859-1"?>
... <test>
...     <Users>
...         <fun25>
...             <limits>
...                 <total>0KiB</total>
...                 <kbps>0</kbps>
...                 <rps>1.3</rps>
...                 <connections>0</connections>
...             </limits>
...             <usages>
...                 <total>16738211KiB</total>
...                 <kbps>237.15</kbps>
...                 <rps>1.3</rps>
...                 <connections>0</connections>
...             </usages>
...             <time_to_refresh>never</time_to_refresh>
...             <limit_exceeded_URL>none</limit_exceeded_URL>
...         </fun25>
...     </Users>
... </test>
... """
>>> a
'<?xml version="1.0" encoding="iso-8859-1"?>\n<test>\n    <Users>\n        <fun25>\n            <limits>\n                <total>0KiB</total>\n                <kbps>0</kbps>\n                <rps>1.3</rps>\n                <connections>0</connections>\n            </limits>\n            <usages>\n                <total>16738211KiB</total>\n                <kbps>237.15</kbps>\n                <rps>1.3</rps>\n                <connections>0</connections>\n            </usages>\n            <time_to_refresh>never</time_to_refresh>\n            <limit_exceeded_URL>none</limit_exceeded_URL>\n        </fun25>\n    </Users>\n</test>\n'
>>>

    @pablogsal
    Copy link
    Member

    New changeset 5ec91f7 by Pablo Galindo in branch 'master':
    bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860)
    5ec91f7

    @miss-islington
    Copy link
    Contributor

    New changeset b2e281a by Miss Islington (bot) in branch '3.8':
    bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860)
    b2e281a

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @corona10 @pablogsal @miss-islington