classification
Title: Buffer overflow in bad_single_statement (Parser/pegen.c)
Type: Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lys.nikolaou, miss-islington, pablogsal
Priority: normal Keywords: patch

Created on 2020-10-25 21:11 by pablogsal, last changed 2020-10-25 23:25 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22978 merged pablogsal, 2020-10-25 21:13
PR 22979 merged miss-islington, 2020-10-25 23:04
Messages (5)
msg379607 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-10-25 21:11
Building Python with the address sanitizer and executing ./python -m test test_repl -v shows this:

===========================================================
==26216==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200003990f at pc 0x5625406527b2 bp 0x7ffe8d621910 sp 0x7ffe8d621900
READ of size 1 at 0x60200003990f thread T0
    #0 0x5625406527b1 in bad_single_statement Parser/pegen.c:993
    #1 0x5625406527b1 in _PyPegen_run_parser Parser/pegen.c:1136
    #2 0x5625406529b8 in _PyPegen_run_parser_from_file_pointer Parser/pegen.c:1179
    #3 0x5625403f1945 in PyRun_InteractiveOneObjectEx Python/pythonrun.c:208
    #4 0x5625403f3cd9 in PyRun_InteractiveLoopFlags Python/pythonrun.c:115
    #5 0x5625403f4e3f in PyRun_AnyFileExFlags Python/pythonrun.c:74
    #6 0x562540181792 in pymain_run_stdin Modules/main.c:512
    #7 0x562540181792 in pymain_run_python Modules/main.c:601
    #8 0x562540182904 in Py_RunMain Modules/main.c:677
    #9 0x562540182904 in pymain_main Modules/main.c:707
    #10 0x562540182904 in Py_BytesMain Modules/main.c:731
    #11 0x7f25297aa151 in __libc_start_main (/usr/lib/libc.so.6+0x28151)
    #12 0x56254017f0bd in _start (/home/pablogsal/github/python/master/python+0x1640bd)

0x60200003990f is located 1 bytes to the left of 2-byte region [0x602000039910,0x602000039912)
allocated by thread T0 here:
    #0 0x7f2529b70459 in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x562540553416 in tok_nextc Parser/tokenizer.c:872
    #2 0x562540554eb7 in tok_get Parser/tokenizer.c:1190
    #3 0x562540559590 in PyTokenizer_Get Parser/tokenizer.c:1839
    #4 0x56254064f6dd in _PyPegen_fill_token Parser/pegen.c:584
    #5 0x5625406934a7 in statement_newline_rule Parser/parser.c:1296
    #6 0x5625406934a7 in interactive_rule Parser/parser.c:769
    #7 0x5625406934a7 in _PyPegen_parse Parser/parser.c:24823
    #8 0x562540652382 in _PyPegen_run_parser Parser/pegen.c:1111
    #9 0x5625406529b8 in _PyPegen_run_parser_from_file_pointer Parser/pegen.c:1179
    #10 0x5625403f1945 in PyRun_InteractiveOneObjectEx Python/pythonrun.c:208
    #11 0x5625403f3cd9 in PyRun_InteractiveLoopFlags Python/pythonrun.c:115
    #12 0x5625403f4e3f in PyRun_AnyFileExFlags Python/pythonrun.c:74
    #13 0x562540181792 in pymain_run_stdin Modules/main.c:512
    #14 0x562540181792 in pymain_run_python Modules/main.c:601
    #15 0x562540182904 in Py_RunMain Modules/main.c:677
    #16 0x562540182904 in pymain_main Modules/main.c:707
    #17 0x562540182904 in Py_BytesMain Modules/main.c:731
    #18 0x7f25297aa151 in __libc_start_main (/usr/lib/libc.so.6+0x28151)

SUMMARY: AddressSanitizer: heap-buffer-overflow Parser/pegen.c:993 in bad_single_statement
Shadow bytes around the buggy address:
  0x0c047ffff2d0: fa fa 00 04 fa fa 00 01 fa fa 00 04 fa fa 00 04
  0x0c047ffff2e0: fa fa 00 04 fa fa 00 04 fa fa 00 04 fa fa 00 04
  0x0c047ffff2f0: fa fa 00 04 fa fa 00 04 fa fa 00 04 fa fa 00 07
  0x0c047ffff300: fa fa 00 04 fa fa 00 04 fa fa 06 fa fa fa 00 fa
  0x0c047ffff310: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
=>0x0c047ffff320: fa[fa]02 fa fa fa fd fa fa fa fd fa fa fa fd fa
  0x0c047ffff330: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047ffff340: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047ffff350: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047ffff360: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047ffff370: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==26216==ABORTING
msg379608 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-10-25 21:12
The problem seems to be that when we do

const char *cur = strchr(p->tok->buf, '\n');
...

*(cur - 1) == '\\'

we don't check if "cur" is before at p->tok->buf (and therefore we cannot check the previous char)
msg379609 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-10-25 21:15
Check the errors in this new builder I have prepared:

https://buildbot.python.org/all/#/builders/582/builds/200/steps/5/logs/stdio
msg379617 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-10-25 23:03
New changeset e68c67805e6a4c4ec80bea64be0e8373cc02d322 by Pablo Galindo in branch 'master':
bpo-42150: Avoid buffer overflow in the new parser (GH-22978)
https://github.com/python/cpython/commit/e68c67805e6a4c4ec80bea64be0e8373cc02d322
msg379619 - (view) Author: miss-islington (miss-islington) Date: 2020-10-25 23:25
New changeset 0b290dd2171e745d94f48298cafb2327eb2de17c by Miss Skeleton (bot) in branch '3.9':
bpo-42150: Avoid buffer overflow in the new parser (GH-22978)
https://github.com/python/cpython/commit/0b290dd2171e745d94f48298cafb2327eb2de17c
History
Date User Action Args
2020-10-25 23:25:04miss-islingtonsetmessages: + msg379619
2020-10-25 23:04:02miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request21896
2020-10-25 23:03:53pablogsalsetstatus: open -> closed
resolution: fixed
stage: resolved
2020-10-25 23:03:46pablogsalsetmessages: + msg379617
2020-10-25 21:15:12pablogsalsetmessages: + msg379609
stage: patch review -> (no value)
2020-10-25 21:13:40pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request21895
2020-10-25 21:12:48pablogsalsetmessages: + msg379608
2020-10-25 21:11:23pablogsalcreate