classification
Title: decorator syntax: allow testlist instead of just dotted_name
Type: enhancement Stage:
Components: Interpreter Core Versions: Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Guido.van.Rossum, benjamin.peterson, berker.peksag, dirn, ebarry, eric.araujo, eric.smith, eric.snow, gvanrossum, infinity0, james, jdemeyer, ncoghlan, peyton
Priority: normal Keywords: patch

Created on 2013-11-20 01:27 by james, last changed 2019-01-21 14:36 by jdemeyer.

Files
File name Uploaded Description Edit
decorator-syntax.patch james, 2013-11-20 01:27 review
decorator-syntax.patch james, 2013-11-21 01:36 review
decorator-syntax.patch james, 2013-11-21 02:31 review
Messages (20)
msg203456 - (view) Author: James Powell (james) Date: 2013-11-20 01:27
Decorator syntax currently allows only a dotted_name after the @. As far as I can tell, this was a gut-feeling decision made by Guido. [1]

I spoke with Nick Coghlan at PyTexas about this, and he suggested that if someone did the work, there might be interest in revisiting this restriction.

The attached patch allows any testlist to follow the @.

The following are now valid:

@(lambda x:x)
def f():
    pass

@(spam if p else eggs)
def f():
    pass

@spam().ham().eggs()
def f():
    pass

[1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html
msg203531 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-11-20 20:36
Thanks for this!  Tests should exercise the now-valid syntaxes, which also need documentation.
msg203532 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-11-20 20:37
On second thought, as this patch allows one form that Guido doesn’t want (bar().foo()), maybe there should be a discussion on python-ideas.
msg203538 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-20 22:12
Nice! As a syntax change (albeit a minor one), I believe this will require a PEP for Python 3.5.

I know Guido indicated he was OK with relaxing the current restrictions, but I don't remember exactly where he said it, or how far he was willing to relax them.
msg203539 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2013-11-20 22:15
I don't feel very strongly, but I do think that most of the things the new syntax allows are not improvements -- they make the decorator harder to read. It was intentional to force you to compute a variable before you can use it as a decorator, e.g.

spamify = (spam if p else eggs)

@spamify
def f():
    pass
msg203554 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-11-21 01:34
> they make the decorator harder to read.

I agree.
msg203555 - (view) Author: James Powell (james) Date: 2013-11-21 01:36
I see this as removing a restriction and a special-case from the
decorator syntax (noting, of course, that these were introduced
deliberately.)

In terms of whether the new forms are improvements, my preference is to
leave this up to the judgement of the programmer, moderated of course by
their prevailing coding guide.

I would argue that this change does not force any additional complexity
on the programmer (who is free to take or leave it) or on the
interpreter (- the straightforwardness of the patch corroborates this.)

I would also argue that there are certainly cases where, in the midst of
some large codebase, the dotted_name restriction may seem a bit arbitrary.

This is likely true for:

class Foo:
    def bar(self, func):
        return func

    @staticmethod
    def baz(func):
        return func
	
    @staticmethod
    def quux():
        def dec(func):
            return func
        return dec

# invalid
@Foo().bar
def f(): pass

# valid
@Foo.baz
def f(): pass

# valid
@Foo.quux()
def f(): pass

For completeness' sake, I have attached a patch with an additional unit
test and amended documentation.

Should we proceed with writing a PEP for Python 3.5?
msg203569 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-21 03:32
Yes, a PEP for 3.5 on this will be valuable, whether it's accepted or not
(although I personally favour moving these restrictions out of the compiler
and into the PEP 8 style guide).

If I recall the past python-ideas threads correctly, the main objections to
the current syntax restrictions were:

- you can't look up decorators through a registry by default, since
"@registry[name]" is disallowed
- it's not really a limitation anyway, since a pass through function still
lets you write whatever you want:

    def deco(x): return x

    @deco(registry[name])
    def f(): ...

Now that the precedent of keeping decorator expressions simple has been
well and truly established, simplification of the grammar is the main
reason removing the special casing of decorator syntax from the compilation
toolchain appeals to me.
msg203728 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2013-11-22 07:33
I think the complexity delta in the grammar is exactly 0.
msg203743 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2013-11-22 12:09
While I think that the dotted_name restriction should be relaxed and it should instead be a style guide issue, I have to agree with Benjamin here: the difference in grammar complexity is zero and shouldn't drive the decision.
msg239697 - (view) Author: Ximin Luo (infinity0) Date: 2015-03-31 12:58
Yes, please get rid of this restriction. It's trivial to get around - you don't even need to define your own "pass-through", one already exists in the standard library:

>>> @(lambda: [lambda x: x][0])()
  File "<stdin>", line 1
    @(lambda: [lambda x: x][0])()
     ^
SyntaxError: invalid syntax

>>> from functools import partial as _
>>> @_( (lambda: [lambda x: x][0])() )
... def f(x): return x*x
... 
>>> f(3)
9

I don't know the rational behind disallowing bar().foo(), but it is the use-case I have in mind - something like:

@MainDecoratorFactory(params).tweakedDecorator(tweak_params)
def f(x):
  pass

or even

@MainDecoratorFactory(params).\
 tweakedDecorator(tweak_params)
def f(x):
  pass

It should be no more controversial than chaining decorators.

The alternative with the current restrictions would be tweakedDecorator(MainDecorator(params), tweak_params) which is more ugly and visually separates the "tweak" concepts.

It's not appropriate to merge MainDecoratorFactory and the tweaks together: there are several MainDecoratorFactories taking care of one main concern; they don't care about the tweaks. And vice versa; the tweaks shouldn't care about the main decorators.
msg271744 - (view) Author: Guido van Rossum (Guido.van.Rossum) Date: 2016-07-31 15:41
Nobody has posted a real use case. All the examples are toys. What are the
real use cases that are blocked by this? Readability counts!
msg271750 - (view) Author: Emanuel Barry (ebarry) * (Python triager) Date: 2016-07-31 21:32
TL;DR - Use case is dynamic decorators. Not all of the syntax would make sense, see below.

The main benefit of this feature would be for dynamic decorators (as was evidenced from others in this issue). In a project I contribute to, we use dynamic decorators to set a function as being a command, and we use the object (a wrapper around the function) directly, so we need a bit more boilerplate around the place.

Ultimately, we would definitely use such a feature (just the '@spam().eggs()' part; we'd have no use for the other ones) , but we probably won't notice its absence either; we've worked around it for years, after all.

As far as readability goes, I think allowing only the '@spam().eggs()' version would actually improve readability quite a bit, by reducing the need to separate the decorator assignment in two (or more) parts. I can see the desire to have a '@spam[eggs]' kind of syntax though, again for the dynamic decorators case.

I see no reason to allow creating lambdas or conditional expressions inside decorators expressions. If anything, that'll encourage anti-patterns, whereas e.g. '@spam(value=eggs)' would be more readable, and would let the decorator decide what it wants to do with the value.

And if your decorator can be expressed as a lambda, why don't you put that in/below the function? Surely it's less work than writing the lambda everytime ;)
msg271755 - (view) Author: Guido van Rossum (Guido.van.Rossum) Date: 2016-08-01 00:39
Could you link to an example decorator definition and its call site(s)
that would benefit? I'm lacking the imagination to understand what a
"dynamic decorator" might be. @spam().eggs() is not enough to help me
understand -- I understand quite well what syntax people are
requesting, but I am unclear on what they actually want to do with it.
I worry there's some extreme use of higher-order functions here that
would just get in the way of readability, but a real-world example
might dispell my fear. (That's what I meant when I said "use case".)
msg271758 - (view) Author: Emanuel Barry (ebarry) * (Python triager) Date: 2016-08-01 01:16
Sure, here goes; this is an IRC game bot which I contribute to. Apologies for the long links, it's the only way to make sure this consistently points to the same place regardless of future commits.

The 'cmd' decorator we use is defined at https://github.com/lykoss/lykos/blob/1852bf2c442d707ba0cbc16e8c9e012bcbc4fcc5/src/decorators.py#L67 - we use its __call__ method to add the function to it; see next link.

How it's used: https://github.com/lykoss/lykos/blob/1852bf2c442d707ba0cbc16e8c9e012bcbc4fcc5/src/wolfgame.py#L9113 - ideally, a syntax such as the following would be nice for these definitions:

­@cmd("myrole", <keyword arguments here>).set
def myrole(cli, nick, chan, rest):
    # ... do stuff here ...

Historically (we used an arcane closure-based version that no one understood), we could call that function after directly, like any normal function. Now, though, we have to call it like this: https://github.com/lykoss/lykos/blob/1852bf2c442d707ba0cbc16e8c9e012bcbc4fcc5/src/wolfgame.py#L764

I'd like to state again that, while we'd use this new syntax, we've already worked around this lack of syntax. Whatever comes out of this, we won't be negatively affected, but decorators are meant to bring whatever alters the function right where it starts, so having syntax that eases that would make sense (to me, anyway).
msg271762 - (view) Author: Guido van Rossum (Guido.van.Rossum) Date: 2016-08-01 02:31
OK, so if you wanted to be able to call myrole(...) instead of
myrole.caller, why doesn't cmd.__call__ return self.caller rather than
just self?
msg271764 - (view) Author: Emanuel Barry (ebarry) * (Python triager) Date: 2016-08-01 02:52
We want to be able to access the instance attributes (as is done e.g. here: https://github.com/lykoss/lykos/blob/1852bf2c442d707ba0cbc16e8c9e012bcbc4fcc5/src/wolfgame.py#L9761 ). I realize we can set the attributes directly on the functions, but we've decided to not do that (it's a style thing, really). Although I guess a class method which then returns our desired method could work out for us.

While I still think that this kind of syntax might be useful for dynamic decorators (I know I'd use that when playing with decorators), I'm afraid I'm out of concrete examples to send your way.
msg271767 - (view) Author: Guido van Rossum (Guido.van.Rossum) Date: 2016-08-01 04:29
OK, maybe someone else wants to provide a real-world example.
Otherwise I am really going to close this (again).
msg321307 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2018-07-09 09:39
Real world example where this actually came up:

https://github.com/jupyter-widgets/ipywidgets/issues/430#issuecomment-247016263
msg334141 - (view) Author: Jeroen Demeyer (jdemeyer) * Date: 2019-01-21 14:36
There is again some discussion about this at https://discuss.python.org/t/why-are-some-expressions-syntax-errors/420
History
Date User Action Args
2019-01-21 14:36:18jdemeyersetmessages: + msg334141
2018-07-09 09:39:06jdemeyersetnosy: + jdemeyer
messages: + msg321307
2016-08-01 04:29:45Guido.van.Rossumsetmessages: + msg271767
2016-08-01 02:52:07ebarrysetmessages: + msg271764
2016-08-01 02:31:27Guido.van.Rossumsetmessages: + msg271762
2016-08-01 01:16:11ebarrysetmessages: + msg271758
2016-08-01 00:39:34Guido.van.Rossumsetmessages: + msg271755
2016-07-31 21:32:32ebarrysetnosy: + ebarry

messages: + msg271750
versions: + Python 3.6, - Python 3.5
2016-07-31 15:41:42Guido.van.Rossumsetmessages: + msg271744
2016-07-31 06:18:41berker.peksagsetnosy: + berker.peksag
2015-03-31 12:58:00infinity0setnosy: + infinity0
messages: + msg239697
2013-11-22 12:09:54eric.smithsetmessages: + msg203743
2013-11-22 07:34:00benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg203728
2013-11-21 03:32:27ncoghlansetmessages: + msg203569
2013-11-21 02:31:33jamessetfiles: + decorator-syntax.patch
2013-11-21 01:36:22jamessetfiles: + decorator-syntax.patch

messages: + msg203555
2013-11-21 01:34:13eric.snowsetnosy: + eric.snow
messages: + msg203554
2013-11-20 22:15:47gvanrossumsetnosy: + gvanrossum
messages: + msg203539
2013-11-20 22:12:45ncoghlansetnosy: + Guido.van.Rossum

messages: + msg203538
versions: - Python 3.4
2013-11-20 20:37:34eric.araujosetnosy: + ncoghlan, - nick
2013-11-20 20:37:06eric.araujosetnosy: + nick
messages: + msg203532
2013-11-20 20:36:04eric.araujosetnosy: + eric.araujo
messages: + msg203531
2013-11-20 19:18:26peytonsetnosy: + peyton
2013-11-20 15:55:14dirnsetnosy: + dirn
2013-11-20 02:26:38eric.smithsetnosy: + eric.smith
2013-11-20 01:27:52jamessetfiles: + decorator-syntax.patch
keywords: + patch
2013-11-20 01:27:05jamescreate