I tried to update the reference manual to the current
Python syntax. Some things are still missing, most
notably the yield expression. Detailed description
of changes below. I can also attach the
generated webpages, if someone is interested.

Expressions
===========

List Displays
-------------
Reordered the rules so that the style is
consistent with the rest of the manual. Separated
listmaker into expression_list and
list_comprehension, for better readability.
Replaced "expression_list" between "for" and "in"
with "target_list". See this thread for details:
http://mail.python.org/pipermail/python-dev/2006-April/064264.html

The only thing missing is old_lambdadef.

Generator Expressions
---------------------
Simmilar as above.

Calls
-----
Fixed the latex syntax (somebody forgot to remove
a line when generators were introduced). Replaced
test with expression. Fixed allowed positions for
commas (func(*args,) is not allowed).

Boolean operations
------------------
Restructured the new conditional expression so
that it is more readable.


Simple Statements
=================

Augmented assignment statements
-------------------------------
Removed comments from "productionlist" macro,
since they broke the generated grammar.txt file.
Removed empty groups that are not needed anymore,
since automatic conversion to guillemets was
disabled. Unfortunately the escaped operator
characters would still need manual fixing in the
grammar.txt file.

The print statement
-------------------
Removed all uses of the "optional" macro and
replaced them with sqare brackets, since it broke
the generated grammar.txt file.

The import statement
--------------------
Replaced all invalid uses of name with identifier.
Added relative import notation to the grammar
section.

Description of relative imports is still needed.

The exec statement
------------------
Corrected a minor mistake, since

exec "a = 1" or "a = 2"

is not valid Python syntax.
Added a (commented out) section about a strange feature
(you can already treat exec as a function) that should
IMHO be included in documentation and its use encouraged
over the current notation.


Compound statements
===================

The with statement
------------------
Added missing macro.

Function definition
-------------------
Cleaned up "parameter_list" so that it is correct
and expresses all the restrictions, but is still
easier to understand (I hope).


Still needed
------------
Yield became an expression in version 2.5, but this
is not documented.

Logged In: YES 
user_id=849994

I think the token names in the reference should not be
different from those in python/Grammar/Grammar. Aside from
this, the patch is fine.

Logged In: YES 
user_id=1326842

Token names in reference manual already differ
from those in Grammar file. I only added
new tokens where it helps readability:

 - I've split listmaker into expression_list
   (already present in reference manual and
   often used) and list_comprehension.
 - I added conditional_expression, because
   I thought it helps readability.
 - Differences in function definiton and call
   syntax can't be avoided since Grammar file
   doesn't express all the limitations.
 - Same goes for target_list; Grammar uses
   testlist, but that was one of the problems
   raised in the thread mentioned above.

The biggest problem is that what is known as
"test" in Grammar/Grammar, is "expression"
in the reference manual, and I think I fixed
all parts that didn't take this in cosideration.

I'm attaching two new patches.
reference_manual_updated.diff contains another
fix (removed unneeded markup and comments), but
is otherwise the same as the previous patch.

reference_manual_conservative.diff also contains
this fix, but removes tokens list_comprehension
and conditional_expression.

This means that there are still large differences
between Grammar file and reference manual, most
notably:

Grammar        manual
---------------------------------------------
NAME           identifier
expr           or_expr
test           expression
old_test       test
testlist       expression_list | target_list
testlist_safe  testlist

All of these differences were already present
before my changes. Let me know if you want
them fixed.

Logged In: YES 
user_id=764593

I agree that it would be better if they were consistent.

But does the manual have better names?  If so, alpha is a 
good time to fix the grammar file.

Logged In: YES 
user_id=1326842

I created another patch and updated the previous
two with the following fixes:

 - Disabled an example of the EBNF notation
   in introduction to prevent name clashes
   and inclusion into the generated grammar file.
 - yield can be a bare statement in 2.5.
 - Reintroduced "name" in import statements,
   so that explanation can stay the same.
 - Reformated __future__ import statement,
   but left the latex syntax broken so that
   it still won't be included into grammar.txt.
 - Fixed a paragraph about valid __future__
   features.

The new patch is more of an example how hard it
would be to synchronise the names. It is more
or less consistent with the Grammar file, but as
a consequence is completely out of sync with the
surrounding text.
While I would like to see less differences betwen
Grammar and Reference manual, I don't think it is
that easy, since someone would also have to check
the rest of the text and fix all incorrect
references.
I would guess that the same holds for changing
the Grammar file; you would simply have to change
too much code.
I think that the simplest solution is to add the
table from my previous comment to the PEP 306.

Logged In: YES 
user_id=1326842

Attaching a patch updated to the current trunk with
two minor fixes:
 - I reverted the changes to the __future__ statement
   because they broke the output;
 - fixed a mistake in the function definition section;
   I didn't know that ``def f(a, (b, c)=(1, 2)): pass``
   is valid syntax.

Logged In: YES 
user_id=849994

The testlist in list comprehensions ("testlist_safe" in
Grammar/Grammar) is different from the normal testlist. Does
your patch address this?

Logged In: YES 
user_id=1326842

Yes, the only thing missing was old lambda. The new patch
('reference_manual_old_lambda') includes it.

You can probably look at this/complete it and apply it yourself now ;)

Ok, here is the final version. There was a whole
section missing (generator expressions), so it
took some time to complete.  I also removed
the function call and function definition changes,
because I couldn't understand them anymore :). Those
changes were more or less cosmetical anyway.

I'll leave the patch open until tomorrow, and then
check it in if there won't be any objections.


File Added: reference_manual_final.diff

Commited as revision 54559, 54560 (2.5).