Use After Free in python3.11 (commit  2ab27c4af4ddf752)
Steps to reproduce:

1) ./configure --with-address-sanitizer
2) make
3) ./python <input>

I attach some of the input  that lead to the undefined behavior

For the complete description you can find the asan report here:

==1082579==ERROR: AddressSanitizer: heap-use-after-free on address 0x626000045a40 at pc 0x000000735155 bp 0x7fffffffbed0 sp 0x7fffffffbec8
READ of size 8 at 0x626000045a40 thread T0
    #0 0x735154 in ascii_decode /home/elmanto/ddg/other_targets/cpython/Objects/unicodeobject.c:5091:28
    #1 0x735154 in unicode_decode_utf8 /home/elmanto/ddg/other_targets/cpython/Objects/unicodeobject.c:5158:10
    #2 0xc98381 in syntaxerror /home/elmanto/ddg/other_targets/cpython/Parser/tokenizer.c:1087:15
    #3 0xc8d616 in tok_get /home/elmanto/ddg/other_targets/cpython/Parser/tokenizer.c
    #4 0xc8696b in PyTokenizer_Get /home/elmanto/ddg/other_targets/cpython/Parser/tokenizer.c:1884:18
    #5 0xead74c in _PyPegen_check_tokenizer_errors /home/elmanto/ddg/other_targets/cpython/Parser/pegen.c:1260:17
    #6 0xead74c in _PyPegen_run_parser /home/elmanto/ddg/other_targets/cpython/Parser/pegen.c:1292:17
    #7 0xeaebca in _PyPegen_run_parser_from_file_pointer /home/elmanto/ddg/other_targets/cpython/Parser/pegen.c:1377:14
    #8 0xc83a91 in _PyParser_ASTFromFile /home/elmanto/ddg/other_targets/cpython/Parser/peg_api.c:26:12
    #9 0xa0abf1 in pyrun_file /home/elmanto/ddg/other_targets/cpython/Python/pythonrun.c:1197:11
    #10 0xa0abf1 in _PyRun_SimpleFileObject /home/elmanto/ddg/other_targets/cpython/Python/pythonrun.c:455:13
    #11 0xa09b19 in _PyRun_AnyFileObject /home/elmanto/ddg/other_targets/cpython/Python/pythonrun.c:89:15
    #12 0x4dfe94 in pymain_run_file_obj /home/elmanto/ddg/other_targets/cpython/Modules/main.c:353:15
    #13 0x4dfe94 in pymain_run_file /home/elmanto/ddg/other_targets/cpython/Modules/main.c:372:15
    #14 0x4dfe94 in pymain_run_python /home/elmanto/ddg/other_targets/cpython/Modules/main.c:587:21
    #15 0x4dfe94 in Py_RunMain /home/elmanto/ddg/other_targets/cpython/Modules/main.c:666:5
    #16 0x4e154c in pymain_main /home/elmanto/ddg/other_targets/cpython/Modules/main.c:696:12
    #17 0x4e1874 in Py_BytesMain /home/elmanto/ddg/other_targets/cpython/Modules/main.c:720:12
    #18 0x7ffff7a2e0b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
    #19 0x43501d in _start (/home/elmanto/ddg/other_targets/cpython/python+0x43501d)

0x626000045a40 is located 2368 bytes inside of 10560-byte region [0x626000045100,0x626000047a40)
freed by thread T0 here:
    #0 0x4ada79 in realloc (/home/elmanto/ddg/other_targets/cpython/python+0x4ada79)
    #1 0x638e61 in PyMem_RawRealloc /home/elmanto/ddg/other_targets/cpython/Objects/obmalloc.c:602:12
    #2 0x638e61 in _PyObject_Realloc /home/elmanto/ddg/other_targets/cpython/Objects/obmalloc.c:2339:12

previously allocated by thread T0 here:
    #0 0x4ada79 in realloc (/home/elmanto/ddg/other_targets/cpython/python+0x4ada79)
    #1 0x638e61 in PyMem_RawRealloc /home/elmanto/ddg/other_targets/cpython/Objects/obmalloc.c:602:12
    #2 0x638e61 in _PyObject_Realloc /home/elmanto/ddg/other_targets/cpython/Objects/obmalloc.c:2339:12

SUMMARY: AddressSanitizer: heap-use-after-free /home/elmanto/ddg/other_targets/cpython/Objects/unicodeobject.c:5091:28 in ascii_decode
Shadow bytes around the buggy address:
  0x0c4c80000af0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b10: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c4c80000b40: fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd
  0x0c4c80000b50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c4c80000b90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==1082579==ABORTING

Lysandros and Pablo, this *only* occurs when the lexer is reading directly from a file, not when it's reading the same source code from a (bytes) string. All examples are syntax errors (some raise ValueError in the parser).

Here is a smaller reproducer:

x = "ijosdfsd\
def blech():
    pass

This seems to be an error with:

commit a698d52c3975c80b45b139b2f08402ec514dce75
Author: Batuhan Taskaya <isidentical@gmail.com>
Date:   Thu Jan 21 00:38:47 2021 +0300

    bpo-40176: Improve error messages for unclosed string literals (GH-19346)



    Automerge-Triggered-By: GH:isidentical

Batuhan, can you take a look?

I think this should fix the issue, but someone should validate this:

diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 6002f3e05a..1c28737183 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1084,17 +1084,16 @@ syntaxerror(struct tok_state *tok, const char *format, ...)
         goto error;
     }

-    errtext = PyUnicode_DecodeUTF8(tok->line_start, tok->cur - tok->line_start,
+    errtext = PyUnicode_DecodeUTF8(tok->buf, tok->inp - tok->buf,
                                    "replace");
     if (!errtext) {
         goto error;
     }
     int offset = (int)PyUnicode_GET_LENGTH(errtext);
-    Py_ssize_t line_len = strcspn(tok->line_start, "\n");
-    if (line_len != tok->cur - tok->line_start) {
+    Py_ssize_t line_len = strcspn(tok->buf, "\n");
+    if (line_len != tok->buf - tok->inp) {
         Py_DECREF(errtext);
-        errtext = PyUnicode_DecodeUTF8(tok->line_start, line_len,
-                                       "replace");
+        errtext = PyUnicode_DecodeUTF8(tok->buf, line_len, "replace");
     }
     if (!errtext) {
         goto error;

This affects 3.10 as well

Ok, found the problem, we are not resetting the multi-line-start pointer when we are reallocating the tokenizer buffers.

New changeset a342cc5891dbd8a08d40e9444f2e2c9e93258721 by Pablo Galindo in branch 'main':
bpo-44396: Update multi-line-start location when reallocating tokenizer buffers (GH-26676)
https://github.com/python/cpython/commit/a342cc5891dbd8a08d40e9444f2e2c9e93258721

alessandro mantovani, one question, how did you generate the crash scripts?

New changeset d03f342a8389f1ea9100efb0d1a205601e607254 by Miss Islington (bot) in branch '3.10':
bpo-44396: Update multi-line-start location when reallocating tokenizer buffers (GH-26676) (GH-26695)
https://github.com/python/cpython/commit/d03f342a8389f1ea9100efb0d1a205601e607254

Fuzzing experimental techniques, but then I observed the same behavior was happening with vanilla afl++. As a starting queue I used the *.py files that I found in the repo under ‘test’ or so 

Best 

Alessandro Mantovani 

Inviato da iPhone

> Il giorno 12.06.2021, alle ore 19:57, Pablo Galindo Salgado <report@bugs.python.org> ha scritto:
> 
> 
> Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:
> 
> alessandro mantovani, one question, how did you generate the crash scripts?
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44396>
> _______________________________________