classification
Title: Eval with too high string multiplication crashes newer Python versions
Type: crash Stage: patch review
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: BTaskaya, Erik-Lamers1, corona10, hongweipeng, methane, pmpp, serhiy.storchaka, stestagg
Priority: normal Keywords: patch

Created on 2020-12-09 11:05 by Erik-Lamers1, last changed 2021-04-26 16:29 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 23744 merged serhiy.storchaka, 2020-12-12 12:31
PR 25634 open serhiy.storchaka, 2021-04-26 16:29
Messages (5)
msg382791 - (view) Author: Erik Lamers (Erik-Lamers1) Date: 2020-12-09 11:05
For Python version 3.7 and above the following statement will end up in a segfault.

eval("1 + 100"*1000000)

Whereas Python versions 3.6 and below would tread this as a Recursion error.
msg382844 - (view) Author: Steve Stagg (stestagg) Date: 2020-12-10 21:04
Looks like it was introduced by https://github.com/python/cpython/commit/7ea143ae795a9fd57eaccf490d316bdc13ee9065:

bpo-29469: Move constant folding to AST optimizer (GH-2858)
msg382849 - (view) Author: Steve Stagg (stestagg) Date: 2020-12-10 21:34
In python 3.7/8, It's a stack overflow in the constant folding code.

On master, the overflow seems to come out of validate_expr.c.

* thread #1, name = 'python3', stop reason = signal SIGSEGV: invalid address (fault address: 0x7fffff7feff8)
    frame #0: 0x00005555557aadba python3`validate_expr(exp=0x00005555602617c0, ctx=Load) at ast.c:224:16
   221          }
   222          return validate_exprs(exp->v.BoolOp.values, Load, 0);
   223      case BinOp_kind:
-> 224          return validate_expr(exp->v.BinOp.left, Load) &&
   225              validate_expr(exp->v.BinOp.right, Load);
   226      case UnaryOp_kind:
   227          return validate_expr(exp->v.UnaryOp.operand, Load);


300,000 ish stack frames of this:

    frame #70832: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150af40, ctx=Load) at ast.c:224:16
    frame #70833: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b050, ctx=Load) at ast.c:224:16
    frame #70834: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b160, ctx=Load) at ast.c:224:16
    frame #70835: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b270, ctx=Load) at ast.c:224:16
    frame #70836: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b380, ctx=Load) at ast.c:224:16
    frame #70837: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b490, ctx=Load) at ast.c:224:16
    frame #70838: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b5a0, ctx=Load) at ast.c:224:16
    frame #70839: 0x00005555557aadbf python3`validate_expr(exp=0x000055556150b6b0, ctx=Load) at ast.c:224:16


On the one hand, pure python code should never segfault, on the other hand, `eval`ling untrusted input has bigger problems than a segfault on carefully crafted input.
msg382910 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-12-12 12:44
This is known issue, but interesting that the cause of the crash is different in 3.7-3.8 and 3.9+.

PR 23744 adds recursion checks in the AST validator and optimizer similar to the checks in the symtable. It should not break any existing code because too deep AST tree did not pass checks in the symtable in any case.

But it does not solve all problems. A compound statement with too many "elif"s is still crashed because the new parser uses recursion in C to parse it (elif_stmt_rule). I think it should be a separate issue.
msg391850 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-04-25 10:38
New changeset face87c94e67ad9c72b9a3724f112fd76c1002b9 by Serhiy Storchaka in branch 'master':
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744)
https://github.com/python/cpython/commit/face87c94e67ad9c72b9a3724f112fd76c1002b9
History
Date User Action Args
2021-04-26 16:29:48serhiy.storchakasetpull_requests: + pull_request24328
2021-04-25 10:45:04serhiy.storchakalinkissue43664 superseder
2021-04-25 10:38:05serhiy.storchakasetmessages: + msg391850
2020-12-14 22:30:14BTaskayasetnosy: + BTaskaya
2020-12-12 12:44:04serhiy.storchakasetmessages: + msg382910
2020-12-12 12:31:35serhiy.storchakasetkeywords: + patch
nosy: + serhiy.storchaka

pull_requests: + pull_request22602
stage: patch review
2020-12-11 10:12:29hongweipengsetnosy: + hongweipeng
2020-12-11 00:43:12corona10setnosy: + methane
2020-12-11 00:42:52corona10setnosy: + corona10
2020-12-10 21:34:01stestaggsetmessages: + msg382849
2020-12-10 21:04:19stestaggsetnosy: + stestagg
messages: + msg382844
2020-12-09 12:01:06pmppsetnosy: + pmpp
2020-12-09 11:06:31Erik-Lamers1settitle: Eval with two high string multiplication crashes newer Python versions -> Eval with too high string multiplication crashes newer Python versions
2020-12-09 11:05:21Erik-Lamers1setversions: + Python 3.7, Python 3.8, Python 3.9
2020-12-09 11:05:07Erik-Lamers1create