diff --git a/compiler.rst b/compiler.rst --- a/compiler.rst +++ b/compiler.rst @@ -41,8 +41,8 @@ The grammar file for Python can be found in Grammar/Grammar with the -numeric value of grammar rules are stored in Include/graminit.h. The +numeric value of grammar rules stored in Include/graminit.h. The numeric values for types of tokens (literal tokens, such as ``:``, -numbers, etc.) are kept in Include/token.h). The parse tree made up of -``node *`` structs (as defined in Include/node.h). +numbers, etc.) are kept in Include/token.h). The parse tree is made up +of ``node *`` structs (as defined in Include/node.h). Querying data from the node structs can be done with the following macros (which are all defined in Include/node.h): @@ -65,14 +65,14 @@ retrieve the line number of the source code that led to the creation of the parse rule; defined in Python/ast.c -To tie all of this example, consider the rule for 'while':: +For example, consider the rule for 'while':: while_stmt: 'while' test ':' suite ['else' ':' suite] The node representing this will have ``TYPE(node) == while_stmt`` and -the number of children can be 4 or 7 depending on if there is an 'else' -statement. To access what should be the first ':' and require it be an -actual ':' token, `(REQ(CHILD(node, 2), COLON)``. +the number of children can be 4 or 7 depending on whether there is an +'else' statement. ``(REQ(CHILD(node, 2), COLON)`` can be used to access +what should be the first ``:`` and require it be an actual ``:`` token. Abstract Syntax Trees (AST) @@ -169,7 +169,7 @@ PyArena_New() will create a new arena. The returned PyArena structure will store pointers to all memory given to it. This does the bookkeeping of what memory needs to be freed when the compiler is finished with the memory it used. -That freeing is done with PyArena_Free(). This needs to only be called in +That freeing is done with PyArena_Free(). This only needs to be called in strategic areas where the compiler exits. As stated above, in general you should not have to worry about memory @@ -210,12 +210,12 @@ The functions called to generate AST nodes from the parse tree all have the -name ast_for_xx where xx is what the grammar rule that the function handles +name ast_for_xx where xx is the grammar rule that the function handles (alias_for_import_name is the exception to this). sequence of AST nodes stored in asdl_seq structs. Function and macros for creating and using ``asdl_seq *`` types as found -in Python/asdl.c and Include/asdl.h: +in Python/asdl.c and Include/asdl.h are as follows: ``asdl_seq_new()`` @@ -386,19 +386,12 @@ Finally, you need to introduce the use of the new bytecode. Altering Python/compile.c and Python/ceval.c will be the primary places to change. -But you will also need to change the 'compiler' package. The key files -to do that are Lib/compiler/pyassem.py and Lib/compiler/pycodegen.py . -If you make a change here that can affect the output of bytecode that -is already in existence and you do not change the magic number constantly, make -sure to delete your old .py(c|o) files! Even though you will end up changing -the magic number if you change the bytecode, while you are debugging your work -you will be changing the bytecode output without constantly bumping up the -magic number. This means you end up with stale .pyc files that will not be -recreated. Running -``find . -name '*.py[co]' -exec rm -f {} ';'`` should delete all .pyc files you -have, forcing new ones to be created and thus allow you test out your new -bytecode properly. +Any time you make changes that affect the output of bytecode you will need to +update the magic number as well, otherwise Python will reuse your old .pyc +files and you will not be able to test your new bytecode properly. If you do +not wish to constantly bump the magic number you can just delete the old .pyc +files each time by running ``find . -name '*.py[co]' -exec rm -f {} ';'``. Code Objects @@ -435,8 +428,8 @@ Python-ast.c Creates C structs corresponding to the ASDL types. Also - contains code for marshaling AST nodes (core ASDL types have - marshaling code in asdl.c). "File automatically generated by + contains code for marshalling AST nodes (core ASDL types have + marshalling code in asdl.c). "File automatically generated by Parser/asdl_c.py". This file must be committed separately after every grammar change is committed since the __version__ value is set to the latest grammar change revision number. @@ -444,7 +437,7 @@ asdl.c Contains code to handle the ASDL sequence type. Also has code to handle marshalling the core ASDL types, such as number and - identifier. used by Python-ast.c for marshaling AST nodes. + identifier. used by Python-ast.c for marshalling AST nodes. ast.c Converts Python's parse tree into the abstract syntax tree. @@ -504,16 +497,6 @@ opcode.py One of the files that must be modified if Include/opcode.h is. - compiler/ - - pyassem.py - One of the files that must be modified if Include/opcode.h is - changed. - - pycodegen.py - One of the files that must be modified if Include/opcode.h is - changed. - Known Compiler-related Experiments ----------------------------------