This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: 3.11 exec raises SystemError instead of SyntaxError on char after line continuation
Type: behavior Stage: resolved
Components: Versions: Python 3.11, Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Zac Hatfield-Dodds, aroberge, lys.nikolaou, mbussonn, miss-islington, pablogsal, pewscorner
Priority: normal Keywords: patch

Created on 2021-11-06 13:10 by pewscorner, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 29550 merged pablogsal, 2021-11-14 00:03
PR 29551 merged miss-islington, 2021-11-14 01:07
PR 29552 merged pablogsal, 2021-11-14 01:19
Messages (12)
msg405860 - (view) Author: PEW's Corner (pewscorner) * Date: 2021-11-06 13:10
When exec'ing code with an illegal character after the line continuation character '\', python 3.11.0a2 raises this strange exception:

  SystemError: Negative size passed to PyUnicode_New

Here's an example where '#' is the illegal character:


I expected the following exception (which is raised by Python 3.10.0, and also by Python 3.11.0a2 when using eval instead of exec, or when the string content is put into its own file and run as a normal script):

  SyntaxError: unexpected character after line continuation character
msg406188 - (view) Author: Zac Hatfield-Dodds (Zac Hatfield-Dodds) * Date: 2021-11-12 05:37
It looks like this also affects Python 3.9.8, which makes me very suspicious of as the probable cause.

See and for reports.
msg406223 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 17:45
Yep, this also break IPython command completions when users press tab in some places in multiline code.

88f4ec88e282bf861f0af2d237e9fe28fbc8deac is the first offending commit for me in the 3.9 branch:

    [3.9] bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993) (#29071)

    There are two errors that this commit fixes:

    * The parser was not correctly computing the offset and the string
      source for E_LINECONT errors due to the incorrect usage of strtok().
    * The parser was not correctly unwinding the call stack when a tokenizer
      exception happened in rules involving optionals ('?', [...]) as we
      always make them return valid results by using the comma operator. We
      need to check first if we don't have an error before continuing..
    (cherry picked from commit a106343f632a99c8ebb0136fa140cf189b4a6a57)

    Co-authored-by: Pablo Galindo Salgado <>

    NOTE: unlike the cherry-picked original, this commit points at a crazy location
    due to a bug in the tokenizer that required a big refactor in 3.10 to fix.
    We are leaving as-is for 3.9.
msg406224 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 17:54
Note that with the following example, exec and CommandCompiler return a different error:

    from codeop import CommandCompiler

    # identical errors
    #exec('1,\\#\n2') SystemError
    #CommandCompiler()('1,\\#\n2', symbol='exec') SystemError

    # one is syntax the other is System.
    exec('a \ \n')  # SyntaxError
    CommandCompiler()('a \ \n', symbol='exec') # SystemError
msg406235 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-12 19:22
I will be able to take a look later in the weekend, so if someone is able to give it a go before, that would be great.
msg406236 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-12 19:32
> I will be able to take a look later in the weekend

Please take some time for yourself, I added you as you were the original author of the patch to make you aware. I believe most projects can workaround that for now.
msg406301 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-14 01:06
New changeset 25835c518aa7446f3680b62c1fb43827e0f190d9 by Pablo Galindo Salgado in branch 'main':
bpo-45738: Fix computation of error location for invalid continuation (GH-29550)
msg406303 - (view) Author: miss-islington (miss-islington) Date: 2021-11-14 01:30
New changeset bf26a6da7aaedb526c9eb1cb56b0e46d1c10384c by Miss Islington (bot) in branch '3.10':
bpo-45738: Fix computation of error location for invalid continuation (GH-29550)
msg406304 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-14 01:31
This should be fixed by now, but please, check if everything is in order in your respective test suites and confirm here.
msg406305 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-11-14 01:47
New changeset 142fcb40b6e460fa9b4a89fe9846b1ce4176354e by Pablo Galindo Salgado in branch '3.9':
bpo-45738: Fix computation of error location for invalid continuation characters in the parser (GH-29550) (GH-29552)
msg406322 - (view) Author: PEW's Corner (pewscorner) * Date: 2021-11-14 13:39
Seems to work. Thanks!
msg406329 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2021-11-14 18:39
Works for me as well. Thanks.
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89901
2021-11-14 18:39:21mbussonnsetmessages: + msg406329
2021-11-14 13:59:48pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-11-14 13:39:18pewscornersetmessages: + msg406322
2021-11-14 01:47:35pablogsalsetmessages: + msg406305
2021-11-14 01:31:39pablogsalsetmessages: + msg406304
2021-11-14 01:30:10miss-islingtonsetmessages: + msg406303
2021-11-14 01:19:47pablogsalsetpull_requests: + pull_request27801
2021-11-14 01:07:03miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27800
2021-11-14 01:06:49pablogsalsetmessages: + msg406301
2021-11-14 00:03:59pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27799
2021-11-12 19:32:15mbussonnsetmessages: + msg406236
2021-11-12 19:30:37pablogsalsetnosy: + lys.nikolaou, BTaskaya
2021-11-12 19:22:50pablogsalsetmessages: + msg406235
2021-11-12 19:08:34arobergesetnosy: + aroberge
2021-11-12 17:54:34mbussonnsetmessages: + msg406224
2021-11-12 17:45:41mbussonnsetnosy: + pablogsal
2021-11-12 17:45:20mbussonnsetnosy: + mbussonn
messages: + msg406223
2021-11-12 05:37:19Zac Hatfield-Doddssetnosy: + Zac Hatfield-Dodds

messages: + msg406188
versions: + Python 3.9
2021-11-06 13:10:52pewscornercreate