> Every piece of code that uses either lib2to3 or a parser derived from it (including parso and LibCST) will eventually not be able to upgrade the parser because PEG can handle grammars that LL(k) can't.

Since these projects are external, depending on the functionality they are free-to-roll their own parser implementations or make hacks to pass away things. Or fork the Grammar/python.gram to preserve all tokens and generate a Python parser from it.


> If people are interested in using ast.parse extensions as a replacement for lib2to3, I suggest discussing at

I don't quite get what you are proposing here, 

>I propose implementing an optional pass over the parse tree that records lib2to3's "prefix" with each leaf node. The interface would be something like:

How would you do that? By augmenting the AST with the information retrieved from tokens? If so, check this out; https://github.com/leo-editor/leo-editor/blob/master/leo/core/leoAst.py and asttokens.

Also, please move the discussion to somewhere else (like discuss.python.org etc.) since this is not the ideal place to discuss and people might be distracted! (feel free to cc me where you move the discussion)