classification
Title: Segmentation fault in running ast.literal_eval() with large expression size.
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.10, Python 3.9, Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: rhettinger, stestagg, terry.reedy, xxm
Priority: normal Keywords:

Created on 2020-12-22 10:17 by xxm, last changed 2021-01-06 04:21 by rhettinger.

Messages (5)
msg383577 - (view) Author: Xinmeng Xia (xxm) Date: 2020-12-22 10:17
Calling function ast.literal_eval() with large size can cause a segmentation fault in Python 3.5 -3.10. Please check the following two examples. The example 1 works as expected, while the second one triggers segmentation fault on Python 3.5,3.6,3.7,3.8,3.9,3.10. The primary difference between these two examples lay on the value of "n".


Example 1: 
========================================= 
import ast
mylist = []
n = 100000
print(ast.literal_eval("mylist"+"+mylist"*n))
==========================================

The actual output: value Error on Python 3.5,3.7,3.8,3.9,3.10, Recursive Error on Python 3.6 (as expected)



Example 2:
===================================
import ast
mylist = []
n = 1000000
print(ast.literal_eval("mylist"+"+mylist"*n))
===================================

The actual output: segmentation fault on Python 3.5 - 3.10 (not as expected)


My system information:

>> python3.10 -V
Python 3.10.0a2

>> python3.9 -V
Python 3.9.0rc1

>> python3.8 -V
Python 3.8.0

>> python3.7 -V
Python 3.7.3

>> python3.6 -V
Python 3.6.12

>> uname -v
#73~16.04.1-Ubuntu
msg383592 - (view) Author: Steve Stagg (stestagg) Date: 2020-12-22 14:22
Likely duplicate of Issue42609, Probably fixed by bpo-42609
msg383601 - (view) Author: Steve Stagg (stestagg) Date: 2020-12-22 19:11
Confirmed fixed by https://github.com/python/cpython/pull/23744:

Traceback (most recent call last):
  File "/home/sstagg/tmp/fuzztest/cpython/../test.py", line 4, in <module>
    print(ast.literal_eval("mylist"+"+mylist"*n))
  File "/home/sstagg/tmp/fuzztest/prefix/lib/python3.10/ast.py", line 62, in literal_eval
    node_or_string = parse(node_or_string.lstrip(" \t"), mode='eval')
  File "/home/sstagg/tmp/fuzztest/prefix/lib/python3.10/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
RecursionError: maximum recursion depth exceeded during compilation
msg384433 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021-01-05 21:23
I closed the following by Xinmeng as duplicates of this:
#42713 eval(...) 
#42714 compile(...)
#42715 exec(...)
#42716 ast.parse(...)

This is probably a duplicate of #42609, and should likely be closed also.
msg384467 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-01-06 04:21
I think we really need a customer parser for ast.literal_eval(), one that doesn't have the same constraints and that can be counted on for safe evaluation of untrusted input.
History
Date User Action Args
2021-01-06 04:21:58rhettingersetnosy: + rhettinger
messages: + msg384467
2021-01-05 21:23:10terry.reedysetnosy: + terry.reedy
messages: + msg384433
2020-12-26 01:41:57terry.reedylinkissue42713 superseder
2020-12-26 01:41:26terry.reedylinkissue42714 superseder
2020-12-26 01:41:01terry.reedylinkissue42715 superseder
2020-12-26 01:40:38terry.reedylinkissue42716 superseder
2020-12-22 19:11:18stestaggsetmessages: + msg383601
2020-12-22 14:22:32stestaggsetnosy: + stestagg
messages: + msg383592
2020-12-22 10:17:25xxmcreate