classification
Title: add a AST validator
Type: enhancement Stage: resolved
Components: Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Trundle, alex, aronacher, benjamin.peterson, daniel.urban, eric.araujo, ncoghlan, python-dev
Priority: normal Keywords: patch

Created on 2011-07-16 17:10 by benjamin.peterson, last changed 2011-08-09 21:17 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
ast_validator.patch benjamin.peterson, 2011-07-16 17:10
Messages (7)
msg140505 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-07-16 17:10
The goal of this patch to keep people doing silly things like producing a Try with no finalbody or excepthandlers and segfaulting the compiler in unpleasant ways.
msg140510 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2011-07-16 23:41
I see what you did there :P
msg140514 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-07-17 02:53
2011/7/16 Armin Ronacher <report@bugs.python.org>:
>
> Armin Ronacher <armin.ronacher@active-4.com> added the comment:
>
> I see what you did there :P

Is that a message of approval? :)
msg140911 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-07-22 21:38
From a top level view, the patch adds PyAST_Validate (with helpers) to ast.c and one call to PyAST_Validate in bltinmodule.c in builtin_compile(). The added tests do every silly thing you can think of.

Does PyAST_Validate get invoked for all calls to compile() or just those with an ast as input. To put it another way, does it validate *all* asts, including those generated by the code parser, or just user-constructed asts. If just the latter now, it seems that turning it on for parser asts at least temporarily could test that it does not invalidate valid code and could be used to test an ast-level optimizer.

Two obvious questions: 1) does this address a real problem that has happened in the field or something that just *might* happen; 2) what is the time penalty (compared to the time of compilation). I guess these amount to "Is the maintenance burden on developers and speed burden on users worth the benefit?". In theory, I am in favor of anything that prevents segfaults. Would that ctypes use could be similarly validated (yes, this is a joke;-).

A last question, whose answer might depend on those above: given that the new function is not exposed to users, why not validate as part of compilation, to avoid walking the tree twice?
msg140912 - (view) Author: Alex Gaynor (alex) * (Python committer) Date: 2011-07-22 21:40
1) Yes, it address a real concern that arose during Armin's GSOC project which has been developing a unified template compilation architecture (via the AST module) for Django and Jinja2.
2) Asking speed questions about this is silly IMO, if compiling functions is your bottleneck, your application design is very very broken.
msg141410 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-07-29 19:59
It'd be nice to get this in soon, so phase 2 can begin.
msg141844 - (view) Author: Roundup Robot (python-dev) Date: 2011-08-09 21:17
New changeset 4090dfdf91a4 by Benjamin Peterson in branch 'default':
add a AST validator (closes #12575)
http://hg.python.org/cpython/rev/4090dfdf91a4
History
Date User Action Args
2011-08-09 21:17:24python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg141844

resolution: fixed
stage: patch review -> resolved
2011-07-29 19:59:10benjamin.petersonsetmessages: + msg141410
2011-07-23 12:05:40benjamin.petersonlinkissue12608 dependencies
2011-07-22 22:17:29terry.reedysetnosy: - terry.reedy
2011-07-22 21:40:53alexsetmessages: + msg140912
2011-07-22 21:38:02terry.reedysetnosy: + terry.reedy
messages: + msg140911

type: enhancement
stage: patch review
2011-07-19 12:59:56eric.araujosetnosy: + eric.araujo
2011-07-17 12:05:31Trundlesetnosy: + Trundle
2011-07-17 06:06:09daniel.urbansetnosy: + daniel.urban
2011-07-17 02:53:45benjamin.petersonsetmessages: + msg140514
2011-07-16 23:42:53alexsetnosy: + alex
2011-07-16 23:41:05aronachersetnosy: + aronacher
messages: + msg140510
2011-07-16 17:10:08benjamin.petersoncreate