This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Implementation for PEP 318 using java-style syntax
Type: Stage:
Components: Interpreter Core Versions: Python 2.4
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: anthonybaxter Nosy List: anthonybaxter, gvanrossum, mark_t_russell, mwh
Priority: normal Keywords: patch

Created on 2004-06-25 13:26 by mark_t_russell, last changed 2022-04-11 14:56 by admin. This issue is now closed.

File name Uploaded Description Edit
at-deco.diff--2004-08-01 mark_t_russell, 2004-08-01 13:12 Implementation for PEP 318 using java-style syntax
Messages (12)
msg46218 - (view) Author: Mark Russell (mark_t_russell) Date: 2004-06-25 13:26
This implements function decorators using the
Java-style syntax described in

As the patch changes the grammar, you'll need to
make sure Include/graminit.h and Python/graminit.c
are writable after applying the patch but before
compiling (after a standard CVS checkout they are
read-only, which stops pgen from working).


   - adding @ as a token
   - changing the grammer for funcdef
   - adding com_decorator in compile.c
   - adding (based on Guido's version)
msg46219 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2004-07-26 13:30
Logged In: YES 

After applying this patch, any attempts to use the
newly-built python (I deleted graminit.c and graminit.h, to
make sure they were rebuilt) gives the following on any

*-s*)  CC='gcc -pthread' LDSHARED='gcc -pthread -shared'
OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E
./ -q build;; \
*)  CC='gcc -pthread' LDSHARED='gcc -pthread -shared'
OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E
./ build;; \
Traceback (most recent call last):
  File "./", line 6, in ?
    import sys, os, getopt, imp, re
SystemError: compile_node: unexpected node type
make: *** [sharedmods] Error 1
msg46220 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2004-07-26 13:41
Logged In: YES 

Hm. Pilot error on my part. I had to build once to get
graminit.h and graminit.c rebuilt, then rebuild again (after
nuking all .o files) to get it to work properly. *sigh*
msg46221 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2004-07-26 13:58
Logged In: YES 

I still get a failure in test_transformer after a make
distclean and rebuild and make distclean and rebuild.

FAILED (errors=1)
Traceback (most recent call last):
  File "Lib/test/", line 37, in ?
  File "Lib/test/", line 32, in test_main
line 290, in run_unittest
    run_suite(suite, testclass)
line 275, in run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "Lib/test/", line 16, in
    a = transformer.parse(s)
line 50, in parse
    return Transformer().parsesuite(buf)
line 120, in parsesuite
    return self.transform(parser.suite(text))
line 113, in transform
    return self.compile_node(tree)
line 149, in compile_node
    return self.file_input(node[1:])
line 180, in file_input
    self.com_append_stmt(stmts, node)
line 1025, in com_append_stmt
    result = self.com_node(node)
line 768, in com_node
    return self._dispatch[node[0]](node[1:])
line 252, in stmt
    return self.com_stmt(nodelist[0])
line 1018, in com_stmt
    result = self.lookup_node(node)(node[1:])
line 312, in expr_stmt
    lval = self.com_augassign(nodelist[0])
line 922, in com_augassign
    l = self.com_node(node)
line 768, in com_node
    return self._dispatch[node[0]](node[1:])
KeyError: 268
msg46222 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2004-07-26 14:09
Logged In: YES 

Argh. Our Makefile's ability to Do The Right Thing... is not
good. Rebuilding Lib/ and Lib/ makes this
better. The really should do this automatically.

So - all tests pass. As far as I can see, things still to be
done for this to be in a2:

- Documentation (can wait until a3, I guess)
- More coverage in test_decorator (ditto)

msg46223 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2004-07-26 14:20
Logged In: YES 

Well, PEP 306 is the reference for this sort of thing: also
needed (before 2.4 final) are changes to Lib/compiler and  Mark, are you willing/able to work on these?
msg46224 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2004-07-28 03:22
Logged In: YES 

Before 2.4final I'd like to see some changes to the syntax:

- Instead of @test, it should be somewhat more restricted, 
preferably @ followed by a dotted name followed by an 
optional argument list. @1+2 or @[1,2,3] just doesn't make 
sense IMO.

- I'd like there to always be a newline between the last 
decorator and the 'def' keyword. (This in the sake 
of "greppability" per python-dev discussion.)
msg46225 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2004-07-28 03:39
Logged In: YES 

There's some non-standard C: in compile.c line 5641 there's a 
char* declaration that isn't at the start of a block. Easily 
fixed but should definitely be fixed before releasing a2.
msg46226 - (view) Author: Mark Russell (mark_t_russell) Date: 2004-07-28 17:37
Logged In: YES 

I've added an updated version of the patch, with the
following changes:

     - Applied against CVS head (as of 2004-07-28)
     - Non-standard C removed (see Guido's comment)
     - Newline before the def is now mandatory
     - Grammer for decorator expressions more restricted
     - and test_tokenize updated to handle @ tokens
     - Lib/compiler/ minimally updated

Still to do:

     - Complete work on Lib/compiler (i.e. get it to the
point where it
       produces a correct .pyc for code with decorators).
     - Write docs
     - Work through all the steps in PEP 306
     - Add more regression tests to test_decorator

I can put in some time on this in the next few days if this
patch seems likely
to be applied.

The grammar is now:

     decorator: '@' dotted_name [ '(' [arglist] ')' ]
     decorators: decorator ([NEWLINE] decorator)* NEWLINE
     funcdef: [decorators] 'def' NAME parameters ':' suite

This allows

     @foo @bar
     def f(): pass

but not:

     @foo @bar def f(): paqss


    - Someone should take a look at the "case decorator:"
code in
      symtable_node() in compile.c.  All the tests pass, but
      seems a fragile bit of code to be changing

    - Lib/compiler needs some more work - at present
decorators are
      ignored there.  I'll do some more work on this when I
get time
      (probably this weekend).

    - As noted in the comments here, the build system does
not cope
      automatically with grammar changes.  The steps I used
to apply
      the patch are:

	   cd .../dist/src
           patch -p0 < /path/to/patch-file
	   chmod +w Lib/ Lib/
	   ./python Lib/
	   ./python Lib/
	   make clean
	   make test >& make-test.out

       I'm sure this sequence is non-optimal, but it works
for me, and
       all the tests pass on my Linux system afterwards.	   
msg46227 - (view) Author: Mark Russell (mark_t_russell) Date: 2004-07-31 21:49
Logged In: YES 

I've updated the patch again.  See my previous
comment here for the procedure to apply the patch.

Main changes:

     - Added decorator support to Lib/compiler. now passes when compiled
       with Lib/compiler.  The regression tests
       also pass when built with
       Tools/compiler/, apart from which I assume is due to
       trivial differences in the bytecode output.
     - Added a description of decorators to the
       reference manual.  Changed the docs for
       staticmethod and classmethod in the library
       manual to use decorator syntax (with a pointer
       to the decorator description in the
       reference manual).

     - Added more tests to,
       including some possibly useful example
       decorators (e.g. memoize)

Other changes (not directly related to decorators):
     - I regenerated Lib/compiler/ using
       Tools/compiler/  I'm not sure
       whether is still in use -- I had
       to tweak it a bit to make the output match  I changed to emit the
       classes in alphabetical rather than dict
       scan order to make it easier to diff the
       output against

     - had a broken definition of
       assertEquals - it always returned True if
       called with two arguments.  I renamed this
       to assertEqualsOrIgnored and fixed the bug.
       The working test revealed a test failure in

     - Changed pyclbr to use the full (dotted)
       module name for pyclbr.Class.module attribute

     - currently fails because its
       is_method() method fails to spot staticmethod-
       and classmethod- wrapped methods.  See the
       comment on the new test test_decorators().
       I'll have another look at this next week.
       Note that this is a problem with the test,
       not the module.  For now
       test_decorator could just be deleted.
msg46228 - (view) Author: Mark Russell (mark_t_russell) Date: 2004-08-01 13:10
Logged In: YES 

Ignore the "Issues:" paragraph in my previous
comment - I made is_method() a bit
smarter and the test now passes.  I also added
a file Lib/test/ to use as a
test module rather than
msg46229 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2004-08-02 06:16
Logged In: YES 

Checked in on the trunk. I think I got everything. Thanks
for your work on this!

/cvsroot/python/python/dist/src/Doc/lib/asttable.tex,v  <--
new revision: 1.2; previous revision: 1.1
/cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v  <--
new revision: 1.168; previous revision: 1.167
/cvsroot/python/python/dist/src/Doc/ref/ref7.tex,v  <-- 
new revision: 1.39; previous revision: 1.38
/cvsroot/python/python/dist/src/Grammar/Grammar,v  <--  Grammar
new revision: 1.50; previous revision: 1.49
/cvsroot/python/python/dist/src/Include/graminit.h,v  <-- 
new revision: 2.22; previous revision: 2.21
/cvsroot/python/python/dist/src/Include/node.h,v  <--  node.h
new revision: 2.23; previous revision: 2.22
/cvsroot/python/python/dist/src/Include/token.h,v  <--  token.h
new revision: 2.21; previous revision: 2.20
/cvsroot/python/python/dist/src/Lib/,v  <--
new revision: 1.33; previous revision: 1.32
/cvsroot/python/python/dist/src/Lib/,v  <--
new revision: 1.17; previous revision: 1.16
/cvsroot/python/python/dist/src/Lib/,v  <--
new revision: 1.14; previous revision: 1.13
/cvsroot/python/python/dist/src/Lib/,v  <--
new revision: 1.36; previous revision: 1.35
/cvsroot/python/python/dist/src/Lib/compiler/,v  <--
new revision: 1.24; previous revision: 1.23
new revision: 1.16; previous revision: 1.15
new revision: 1.41; previous revision: 1.40
initial revision: 1.1
initial revision: 1.1
new revision: 1.19; previous revision: 1.18
new revision: 1.22; previous revision: 1.21
 <--  tokenize_tests.txt
new revision: 1.2; previous revision: 1.1
 <--  test_tokenize
new revision: 1.9; previous revision: 1.8
<--  parsermodule.c
new revision: 2.83; previous revision: 2.82
/cvsroot/python/python/dist/src/Parser/tokenizer.c,v  <-- 
new revision: 2.75; previous revision: 2.74
/cvsroot/python/python/dist/src/Python/compile.c,v  <-- 
new revision: 2.310; previous revision: 2.309
/cvsroot/python/python/dist/src/Python/graminit.c,v  <-- 
new revision: 2.36; previous revision: 2.35
<--  ast.txt
new revision: 1.6; previous revision: 1.5
new revision: 1.8; previous revision: 1.7
new revision: 1.5; previous revision: 1.4
Date User Action Args
2022-04-11 14:56:05adminsetgithub: 40449
2004-06-25 13:26:43mark_t_russellcreate