This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author serhiy.storchaka
Recipients benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, terry.reedy, yselivanov
Date 2018-02-10.09:00:06
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1518253206.75.0.467229070634.issue32758@psf.upfronthosting.co.za>
In-reply-to
Content
On Linux the limit is much larger that 2**15. This depends on the stack size, it is smaller on Windows.

The stack is overflowed by recursive call of ast2obj_expr in Python/Python-ast.c. The same problem exists in other recursive AST processing code: optimizing, unparsing. This is why 3.7 can crash in cases when 3.6 was not crashed.

I don't see an easy way of fixing this. The common way is surrounding recursive calls with Py_EnterRecursiveCall()/Py_LeaveRecursiveCall(). But this make the limit too small (1000 by default). In other cases it is enough. The data structures with 1000 levels of nesting are rare. In any case the Python compiler has lower limit on nesting indented blocks. But in this case the linear sequence of binary operators is compiled to deeply nested structure.

I see two hard ways of properly fixing this.

1. Rewrite all AST expression processing code with using an explicit stack instead of function stack. This will complicate a lot of C code too much. Python code should be rewritten too if you don't want to get a RecursionError in corner cases, but for common cases it can be left unmodified.

2. Change binary operations representation, so that a+b+c+d will be represented as ('+', (a, b, c, d)) instead of ('+', ('+', ('+', a, b), c), d). This is backward incompatible change and needs to rewrite any AST processing code (in C and Python). This solution can't be backported. But the resulting code will be simpler than when rewrite them for the first approach.
History
Date User Action Args
2018-02-10 09:00:06serhiy.storchakasetrecipients: + serhiy.storchaka, brett.cannon, terry.reedy, ncoghlan, benjamin.peterson, yselivanov
2018-02-10 09:00:06serhiy.storchakasetmessageid: <1518253206.75.0.467229070634.issue32758@psf.upfronthosting.co.za>
2018-02-10 09:00:06serhiy.storchakalinkissue32758 messages
2018-02-10 09:00:06serhiy.storchakacreate