classification
Title: Add sys.setasthook() to allow to use a custom AST optimizer
Type: Stage:
Components: Versions: Python 3.4
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, haypo
Priority: normal Keywords: patch

Created on 2013-03-22 00:58 by haypo, last changed 2013-03-22 17:09 by brett.cannon.

Files
File name Uploaded Description Edit
setasthook.patch haypo, 2013-03-22 00:58 review
enable_astoptimizer.patch haypo, 2013-03-22 01:09 review
Messages (5)
msg184932 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-03-22 00:58
I wrote an optimizer rewriting the AST to implement various optimizatoions:
https://bitbucket.org/haypo/astoptimizer

To be able to use it, or use any other kind of AST transformation, I propose to add a new sys.setasthook() function to set a custom AST hook.

Attached patch is a proof-of-concept. The patch should be improved: document the function and add write unit tests.

Prototype of the hook:

def asthook(ast, filename):
   # ...
   return new_ast

I don't know if we need to add a compiler option (like _ast.PyCF_ONLY_AST) to skip the hook.
msg184936 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-03-22 01:09
enable_astoptimizer.patch: example of usage of the new function, you can use it to try astoptimizer.
msg184971 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-03-22 15:17
This is unnecessary. I added the source_to_code() method (http://docs.python.org/3.4/library/importlib.html#importlib.abc.SourceLoader.source_to_code) to loaders to explicitly handle this case.
msg184978 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-03-22 15:53
> This is unnecessary.

Your change only concerns imports. eval() and compile() cannot be
hooked using your method.

2013/3/22 Brett Cannon <report@bugs.python.org>:
>
> Brett Cannon added the comment:
>
> This is unnecessary. I added the source_to_code() method (http://docs.python.org/3.4/library/importlib.html#importlib.abc.SourceLoader.source_to_code) to loaders to explicitly handle this case.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue17515>
> _______________________________________
msg184986 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-03-22 17:09
And that's fine as it allows for more explicit control over what gets optimized. Basically if there is a global hook directly into the compiler there needs to be more control over detecting it's being optimized, what to do about .pyo files, etc. At least at the loader level it's much more modular and controlled as to what will and will not get the optimizations.

So if you want to pursue this I think there needs to be more thought than just a global sys hook for this sort of thing.
History
Date User Action Args
2013-03-22 17:09:41brett.cannonsetmessages: + msg184986
2013-03-22 15:53:19hayposetmessages: + msg184978
2013-03-22 15:17:42brett.cannonsetmessages: + msg184971
2013-03-22 01:09:12hayposetfiles: + enable_astoptimizer.patch

messages: + msg184936
2013-03-22 00:58:11haypocreate