Two things about the UTF-8 decode issue. One is you don't need to decode the source for it to be compiled; ast.parse()/compile() will work from bytes objects and thus handle the decoded for you. Two, if you want to decode source files into text, use importlib.util.decode_source().

Since Terry's script shows a complete compilation of the stdlib is very fast, we should make this a proper test. Any resulting test should probably should go into test_compile(). We can have it read all files and those that raise a a SyntaxError or some such exception can simply be skipped as the test is about AST -> code and not source -> AST (although we could theoretically have the test validate the failure with a source -> code compilation as well and verify the same exception is raised). We can also double-check that any AST -> code failure passes under source -> code and then report a failure (basically I'm trying to avoid blacklisting files that are test cases for the compiler overall and are known to fail).