> the idea was proposed purely to "close a gap"

That pinpoints it well. I was just writing a tutorial on implementing custom import hooks, with the idea to show people how easy it to do it in Python. As first step, I explained that it's bad idea to do any transformations on surface representation of a program. At the very least, it should be converted to token stream. But then I found that I need to explain that we need to convert it back, which sounds pretty weird and undermines the idea:

    def xform(token_stream):
        for t in token_stream:
            if t[0] == tokenize.NAME and t[1] == "function":
                yield (tokenize.NAME, "lambda") + t[2:]
            else:
                yield t

    with open(filename, "rb") as f:
        # Fairly speaking, tokenizing just to convert back to string form
        # isn't too efficient, but CPython doesn't offer us a way to parse
        # token stream so far, so we have no choice.
        source = tokenize.untokenize(xform(tokenize.tokenize(f.readline)))
    mod = type(imphook)("")
    exec(source, vars(mod))
    return mod

Having written that comment, I thought I could as well just make one more step and monkey-patch "ast" for parse_tokens() function - I'll need to explain that, but the explanation probably wouldn't sound worse than the explanation above. And then it was just one more step to actually submit patch for ast.parse_tokens(), and that's how this ticket was created!