classification
Title: compilation seg faults on insanely large expressions
Type: crash Stage:
Components: Interpreter Core Versions: Python 3.2, Python 2.7
process
Status: open Resolution: duplicate
Dependencies: Superseder: stack overflow evaluating eval("()" * 30000)
View: 5765
Assigned To: Nosy List: SilentGhost, haypo, ncoghlan, serhiy.storchaka
Priority: normal Keywords:

Created on 2011-03-03 10:56 by ncoghlan, last changed 2012-12-20 21:08 by haypo.

Messages (8)
msg129950 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-03-03 10:56
~/devel/py3k$ ./python -c "compile('1*'*100000+'1', 'broken', 'eval')"
Segmentation fault

Going by the gdb stack trace we're blowing the stack due to the recursive descent in "compiler_visit_expr".
msg129952 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-03-03 11:10
Updated Lib/test/crashers/compiler_recursion.py to refer back to this issue. (As well as making it actually crash again on my system - apparently an expression nested 59k deep wasn't enough to kill the stack here, so I bumped it to 100k)
msg131245 - (view) Author: SilentGhost (SilentGhost) Date: 2011-03-17 13:39
100k is, apparently, not enough on my system (linux2). test_crashers now fails. Are any system-specific details needed?
msg131248 - (view) Author: SilentGhost (SilentGhost) Date: 2011-03-17 13:43
10**6 on the other hand seem to do the job
msg174762 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-11-04 07:47
I've started looking into what would be needed to fix this. The basic problem is that the compilation process involves many recursive operations, but doesn't contain *any* calls to the recursion control functions (http://docs.python.org/3/c-api/exceptions.html#recursion-control).

Files to be investigated:
Python/ast.c
Python/symtable.c
Python/compile.c

Suspicion should fall immediately on any functions in these files which end with "_stmt" and "_expr". The reason as that these are the self-recursive constructs in the Python grammar: statements can contain other statements (via the compound statements with their nested suites) and expressions can contain other expressions.

The symtable analysis also recurses through the block stack via the "analyze_block" function, making that another candidate for flagging with the recursive call functions.
msg174763 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2012-11-04 07:53
One caveat on this idea: it may not be possible to use the standard recursion limiting functions here, since the Python level recursion limit is generally set quite low (1000 by default on my Fedora system).

While this crash *is* a design flaw in our compiler implementation, whatever enforced limit we choose, we run the risk of breaking currently working applications.

Thus, adjusting the target versions to 3.4. The problem still *affects* all versions since 2.5, I'm just indicating that any fix is almost certainly going to be too intrusive to risk in a maintenance release.
msg174765 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-11-04 09:17
Isn't it a duplicate of issue5765?
msg177859 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-12-20 20:51
This bug does not reproduced on 3.3+ more as it was fixed in issue5765. But it is yet here in 2.7 and 3.2.
History
Date User Action Args
2012-12-20 21:08:56hayposetnosy: + haypo
2012-12-20 20:52:16serhiy.storchakasetcomponents: + Interpreter Core
2012-12-20 20:51:33serhiy.storchakasetmessages: + msg177859
versions: + Python 2.7, Python 3.2, - Python 3.4
2012-11-04 09:18:31serhiy.storchakasetsuperseder: stack overflow evaluating eval("()" * 30000)
resolution: duplicate
2012-11-04 09:17:17serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg174765
2012-11-04 07:53:41ncoghlansetmessages: + msg174763
versions: + Python 3.4, - Python 3.2, Python 3.3
2012-11-04 07:47:25ncoghlansetmessages: + msg174762
2011-03-17 13:43:30SilentGhostsetnosy: ncoghlan, SilentGhost
messages: + msg131248
2011-03-17 13:39:58SilentGhostsetnosy: + SilentGhost
messages: + msg131245
2011-03-04 22:55:04terry.reedysettype: crash
versions: + Python 3.2, Python 3.3
2011-03-03 11:10:32ncoghlansetmessages: + msg129952
2011-03-03 10:56:45ncoghlancreate