classification
Title: AST-level Constant folding
Type: enhancement Stage: patch review
Components: Interpreter Core Versions: Python 3.7
process
Status: open Resolution:
Dependencies: 29463 Superseder:
Assigned To: Nosy List: benjamin.peterson, brett.cannon, georg.brandl, haypo, inada.naoki, ncoghlan, serhiy.storchaka, yselivanov
Priority: normal Keywords: patch

Created on 2017-02-07 04:02 by inada.naoki, last changed 2017-02-07 11:32 by inada.naoki.

Files
File name Uploaded Description Edit
ast-constant-folding.patch inada.naoki, 2017-02-07 04:02 review
Messages (4)
msg287186 - (view) Author: INADA Naoki (inada.naoki) * (Python committer) Date: 2017-02-07 04:02
spin off of #11549.
This patch uses code generator to traverse AST.
msg287210 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-07 08:55
I suggest you to look at my AST optimizer, especially the constant folding part:
https://github.com/haypo/fatoptimizer/blob/master/fatoptimizer/const_fold.py

And the unit tests, search for BaseConstantFoldingTests:
https://github.com/haypo/fatoptimizer/blob/master/test_fatoptimizer.py#L1980

IHMO we must have a long test suite on this AST optimizer, because it's common that the AST changes in subtle ways, AST is complex and so we should prevent regressions. You may simply copy my unit tests.

My optimizer implements more optimization: just remove unit tests on cases which you don't want to optimize.
msg287214 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-02-07 09:01
With this change, the Python compiler doesn't emit ast.Num nor ast.Str, right?

If we merge such change, we should prepare projects using AST. There is something like that in pip, but I failed to remind which one :-/ It should be easy: apply your patch, try to install something using pip and see the traceback ;-)
msg287227 - (view) Author: INADA Naoki (inada.naoki) * (Python committer) Date: 2017-02-07 11:32
Do you mean https://github.com/pypa/pip/blob/403e398330c8e841e4633aceda859430f5f7b913/pip/_vendor/distlib/markers.py ?

This doesn't affect, maybe.
Constant folding is executed right before producing bytecode.
It doesn't affect to PyCF_ONLY_AST.

BTW, how do you feel asdl_ct.py?

I feel it's too much only for constant folding, but it is too less for other advanced optimizations.
Actually, original patch in #11549 implements other optimizations
ported from peephole in compile.c.
And most tough part of updating this patch is resolve conflict with this commit.
https://github.com/python/cpython/commit/9cea398e237d4d75c6012e23a33d01b17f5dffcc

I'll try to remove asdl_ct.py and ast_opt.ct in next version.
History
Date User Action Args
2017-02-07 11:32:27inada.naokisetmessages: + msg287227
2017-02-07 09:10:57serhiy.storchakasetnosy: + brett.cannon, georg.brandl, ncoghlan, benjamin.peterson, serhiy.storchaka, yselivanov

type: enhancement
components: + Interpreter Core
stage: patch review
2017-02-07 09:01:57hayposetmessages: + msg287214
2017-02-07 08:55:31hayposetnosy: + haypo
messages: + msg287210
2017-02-07 04:02:52inada.naokisetdependencies: + Add `docstring` field to AST nodes
2017-02-07 04:02:38inada.naokicreate