This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add AutoNumberedEnum to stdlib
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: ethan.furman Nosy List: David Hagen, John Hagen, abarry, barry, eli.bendersky, ethan.furman, kennethreitz, python-dev, rhettinger, veky, vstinner
Priority: normal Keywords: patch

Created on 2016-05-10 01:34 by John Hagen, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue26988.stoneleaf.01.patch ethan.furman, 2016-07-12 01:45 review
issue26988.stoneleaf.02.patch ethan.furman, 2016-07-16 00:25 review
issue26988.stoneleaf.03.patch ethan.furman, 2016-08-02 07:27 review
issue26988.stoneleaf.05.patch ethan.furman, 2016-08-03 01:55
Messages (62)
msg265214 - (view) Author: John Hagen (John Hagen) * Date: 2016-05-10 01:34
I suggest that the AutoNumberedEnum be added to the standard library for the following reasons:

1) Provides a fundamental tool for defining a unique, abstract set of coupled members
2) Avoids boilerplate @enum.unique for a very common use case of enumerations
3) The code already exists in the Python documentation, so it has been vetted at some level

The AutoNumberedEnum also allows the developer to make a clearer distinction
between enumerations whose values have special meaning and those that do not.

Consider:

@enum.unique
class Color(enum.Enum):
    red = 1
    blue = 2
    green = 3

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

With AutoNumberedEnum it's possible to better express the intent that 
the value of Color members does not hold special meaning, while
Shape members do:

class Color(enum.AutoNumberedEnum):
    red = ()
    blue = ()
    green = ()

@enum.unique
class Shape(enum.Enum):
    """Member values denote number of sides."""
    circle = 1
    triangle = 3
    square = 4

For enumerations with many members (10s), there becomes a maintenance
issue when inserting new enumerations into the list:

@enum.unique
class Color(enum.Enum):
    aquamarine = 1
    blue = 2
    fushia = 3
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in having to increment all following members
    ...
    green = 40
    red = 41

Most other languages have support for naming enumerations
without explicitly declaring their values (albeit with 
specialized syntax that makes it cleaner):

C++: http://en.cppreference.com/w/cpp/language/enum
C#: https://msdn.microsoft.com/en-us/library/sbbt4032.aspx
Java: https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html
Rust: https://doc.rust-lang.org/book/enums.html

Currently, a developer can copy the code from the Python docs into
his or her project, or add a dependency on aenum.  I would argue
that it belongs in the standard library.

If there are objections to it being too magical, I would argue
it's already spelled out in the docs, so it's being prescribed
as a solution already.  I think it should be very clear when you
derive from "AutoNumberedEnum" what is going on.

The code is very simple, so I would
hope maintenance would not be difficult.
msg265215 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-05-10 01:58
On the other hand, if you use aenum you can do:

class Color(AutoNumber):
    red
    green
    blue

And isn't that better*?  ;)

* For those in danger of swallowing their tongue over the magic involved: The magic is turned off as soon as any descriptor is defined (property, function, etc.).
msg265216 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-05-10 02:01
Wow Ethan, that's quite interesting.  I'm not sure if I like it or not. :)

It's better than having to assign a dummy value to the enum items, and if we did require that, I'd suggest just throwing away the values for autonumbering.

But not having to specify any values at all is probably better, although it does look weird!

/me goes to look at the magic you're using...
msg265218 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-05-10 02:06
It's in _EnumDict.__getitem__; there's some duplication in __setitem__ for supporting Python 2 (although with 2 you have to use either the empty tuple, or some other handy thing that may go in the __doc__ attribute, for example).
msg265237 - (view) Author: John Hagen (John Hagen) * Date: 2016-05-10 11:17
@Ethan, I hadn't tried to use the aenum AutoNumberEnum that way, but I agree with Barry that I like it.  To me that is the ideal case we should shoot for as I think it's the best* long term and deviate only if practical concerns prevent it.

So I am +1 for empty member assignment and if that is rejected, +1 for = () assignment as at least it is a big step forward.  I feel both solutions already have some "magic", so would lean toward the one that leads to the least amount of boilerplate.

As for the empty assignment, I have played around with something similar before and will throw out one con for it: static analyzers get really confused.  PyCharm, for example, thinks this is has unresolved references in it:

class Color(AutoNumberEnum):
    red
    green
    blue

But the counter point is that if this is in the stdlib, static analyzer authors are much more likely to add a special case for it than if in a "non-official" third party package (PyCharm example: https://youtrack.jetbrains.com/issue/PY-19150).


Another piece of evidence to support inclusion is that Python already provides specialized Enum subclasses (like IntEnum).  I've written a lot of Python code that uses Enums and haven't personally needed IntEnum yet, but would have used an AutoEnum many, many times.

* I am assuming here that a true "enum" keyword is out of the question at this point for Python, which would probably be even better.
msg269476 - (view) Author: John Hagen (John Hagen) * Date: 2016-06-29 11:33
@Ethan/Barry what needs to be done now to accept or reject this for Python 3.6?  Should I propose it onto python-dev?  If accepted, would be nice to get it in before alpha 3 (~2 weeks).  What's nice about this proposal is Ethan has already written the code, so it's just a matter of getting consensus to add it.
msg269521 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-06-29 15:22
I don't really think much more needs to be done other than agree on this tracker issue that it's something we want, and that Ethan's implementation is good enough.  It's been a while since I looked that the code but it seemed good to me before, so if Ethan is happy with it going into the stdlib, so am I.  As for the feature, it *is* a little magical, but not so much that it would be confusing or ambiguous, so I'm +1 on it.

I'll defer to Ethan for the final decision, but if we have tests and documentation, I'd be very happy to see this in 3.6.
msg270039 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-09 13:04
Is this something we want to get in before the next alpha in two days?  Just wanted to bring up the deadline since this may be a feature people want to play around with during the alpha phase.

Ethan, I'm happy to help with documentation or anything else.
msg270048 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-09 15:20
There is one wrinkle with auto-numbering.  Consider the following code:

class Color(AutoNumberEnum):
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

What happens with `property`?

- `property` is looked up in the class namespace
- it's not found, so is given the number 4
- it's then called to handle `cap_name`
- an exception is raised

As far as I can tell there is only one "good" way around this:

- store any names you don't want auto-numbered into an `_ignore_`
  attribute (this only needs to be global and built-in names that
  are used before the first method is defined)

another option is to build a proxy around any found global/built-in objects and decide what to do based on whether those objects are immediately called, but that fails when the object is simply assigned for later use.

So, barring any other ideas to handle this problem, the above example should look like this:

class Color(AutoNumberEnum):
  _ignore_ = 'property'
  red
  green
  blue
  @property
  def cap_name(self):
    return self.name.upper()

Another advantage of using ignore is the ability to have temporary variables automatically discarded:

  class Period(timedelta, Enum):
    '''
    different lengths of time
    '''
    _ignore_ = 'Period i'
    Period = vars()
    for i in range(31):
      Period['day_%d' % i] = i, 'day'
    for i in range(15):
      Period['week_%d' % i] = i*7, 'week'
    for i in range(12):
      Period['month_%d' % i] = i*30, 'month'
    OneDay = day_1
    OneWeek = week_1
    def __new__(self, value, period):
      ...

and the final enumeration does not have the temp variables `Period` nor `i`.

Thoughts?
msg270049 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-07-09 16:04
On Jul 09, 2016, at 03:20 PM, Ethan Furman wrote:

>As far as I can tell there is only one "good" way around this:
>
>- store any names you don't want auto-numbered into an `_ignore_`
>  attribute (this only needs to be global and built-in names that
>  are used before the first method is defined)

Seems reasonable, though why not a dunder, e.g. __ignore__?

>    _ignore_ = 'Period i'

Why space separated names inside a string instead of a sequence of strings?
msg270051 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-09 16:19
A standard feature of Enum is that either space separated strings or a list of strings is accepted any where either is.

_sunder_ names are the ones reserved for Enum use (such as _value_, _name_, and soon _order_ (for py2/py3 compatible code).
msg270054 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-09 17:02
> What happens with `property`?
> 
> - `property` is looked up in the class namespace

Perhaps you've already considered this, I'm not intimately familiar with how classes are parsed and constructed but is it possible to determine if the object is a decorator?  It already determines to stop auto-numbering when it hits the first method, could it stop when it hits the first decorator or method?

Being able to use temporaries is an interesting side effect, but I feel like that would be used less often than a @staticmethod, @property, or @classmethod over a method, in which case it becomes a little more complex.

That being said, I think either solution is valid.
msg270056 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-09 17:11
The problem with testing the type of object a name refers to outside the class is it then becomes more difficult to make that an Enum member:

class AddressType(Enum):
    pobox
    mailbox  # third-party po box
    property

Having to assign a value to `property` pretty much negates the value of the magic.

I'll go with `_ignore_`.
msg270071 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-10 05:41
I need names.  `aenum` already has an `AutoNumberEnum` (the one from the docs, no magic) so I hate to use the same name for the stdlib version with different behavior.

So I either need a different name for the stdlib version, or a different name for the aenum version.

Any ideas?

Hmmm... maybe SequentialEnum for the aenum version...
msg270088 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-10 14:35
Some ideas for the new stdlib class:

BasicEnum - This helps emphasize that it is a simpler version of Enum that doesn't support all of the Enum features (like assigning values). It also helps communicate that if you don't need values this is a better fit.

AutoEnum - This new version (compared with AutoNumberEnum in the docs) does more than just auto number, since it does even the value assignment. Auto helps communicate that this is automatically creating much of the class internals for you.
msg270107 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-10 17:42
I like AutoEnum.

Another auto-related thought:  one of the more common Enum questions on StackOverflow is how to automatically have the value be the stringified version of the name:

class Huh(Enum):
  this
  that
  those

Huh.this.name == Huh.this.value
# True

So the question then becomes: is there a way to easily support both auto-number and auto-string values?

While I don't have the auto-string feature yet in aenum, the system I am using to specify optional settings looks like this:

------
class Color(Enum, settings=AutoNumber):
  red
  ...
------

------
class Color(IntEnum, settings=AutoNumber):
  red
  ...
------

------
class Color(Enum, settings=AutoName):
  red
  ...
------


The other option, of course, is to just stick with the prebuilt method of doing things:

class Color(AutoEnum):
  ...

class Color(AutoEnum, IntEnum):
  ...

class Color(AutoNameEnum):
  ...
msg270117 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-07-10 18:09
On Jul 10, 2016, at 05:42 PM, Ethan Furman wrote:

>class Color(Enum, settings=AutoNumber):
[...]
>class Color(Enum, settings=AutoName):

I guess `settings` would take an AutoType enum.  But that can't also be
autonumbered or it would be autos all the way down. :)
msg270137 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-10 22:26
To me, class Color(AutoEnum) and class Color(AutoEnum, IntEnum) is a little more straightforward.  It makes usage of AutoEnum similar to IntEnum, and I would expect it to be at least as popular as it.

A enum-related side question, since the plan is for this to go into the stdlib, would it also go into enum34 since that is the official back port of the stdlib Enum?
msg270142 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-11 00:27
Yeah, I think the public interface will just be the AutoEnum and AutoNameEnum style.

---

> Will these features go into enum34?

Not sure.  At this point I have the stdlib enum, enum34 enum, and aenum enum.

In terms of capability, aenum is the most advanced, followed by the stdlib enum, and finally enum34 (really the only difference between stdlib and enum34 is the automatic definition order).

The only advantage of enum34 over aenum is if it works in enum34 it will definitely work in the stdlib, whilst aenum has features not in the stdlib (speaking from a user point of view).

So I haven't decided, but at this moment I'm not excited about the prospect.
:(

What I'll probably do is put enum34 in bug-fix only mode.
msg270148 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-07-11 01:25
On Jul 11, 2016, at 12:27 AM, Ethan Furman wrote:

>Not sure.  At this point I have the stdlib enum, enum34 enum, and aenum enum.
>
>In terms of capability, aenum is the most advanced, followed by the stdlib
>enum, and finally enum34 (really the only difference between stdlib and
>enum34 is the automatic definition order).
>
>The only advantage of enum34 over aenum is if it works in enum34 it will
>definitely work in the stdlib, whilst aenum has features not in the stdlib
>(speaking from a user point of view).
>
>So I haven't decided, but at this moment I'm not excited about the prospect.
>:(
>
>What I'll probably do is put enum34 in bug-fix only mode.

It's been useful to have a standalone version of the stdlib module, and in
fact, I maintain the enum34 package in Debian.  However, we only support that
for Python 2 since we don't have to worry about any Python 3 versions before
3.4 (and even there, 3.5 is the default for Stretch and Ubuntu 16.04 LTS).

We do have reverse dependencies for python-enum34, but given that we *really*
want people to port to Python 3, I'm not sure I really care too much any more
about enum34 in Debian.
msg270151 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-11 01:34
That brings up a really good point -- this feature requires the __prepare__ method of the metaclass, so it won't work in Python 2 any way.

So, yeah, bug-fix-mostly mode for enum34.  :)
msg270199 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-11 19:05
If you are constructing your own base Enum type, which would you rather do?

class BaseZeroEnum(Enum):
   "initial integer is 0"
   _start_ = 0
   ...

or

class BaseZeroEnum(Enum, start=0):
   "initial integer is 0"
   ...

?  Oh, and yes if you specify a starting number you also activate the AutoNumber feature.
msg270201 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-07-11 19:25
On Jul 11, 2016, at 07:05 PM, Ethan Furman wrote:

>class BaseZeroEnum(Enum, start=0):
>   "initial integer is 0"
>   ...
>
>?  Oh, and yes if you specify a starting number you also activate the
>AutoNumber feature.

I like the way this one looks.
msg270212 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-11 22:47
>class BaseZeroEnum(Enum, start=0):
>   "initial integer is 0"
>   ...

I also think this looks better.
msg270219 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-12 01:45
Here's the code.  I'll do the doc changes next.
msg270238 - (view) Author: John Hagen (John Hagen) * Date: 2016-07-12 11:52
I like the addition of UniqueEnum.  It's the default use case often.
msg270523 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-16 00:25
Okay, here's an updated patch with the doc changes.

Once the main patch is committed I'm going to reorganize the docs a bit, but that's later.
msg270524 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-07-16 00:28
Oh, and yes, I'll fix the whole 80-column thing.  ;)
msg271811 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-02 07:27
The 80-column thing isn't fixed yet, but I redid the patch:

- removed AutoNameEnum since there was some question of casing, etc
- redid the methodology of auto-generating values:
    instead of adding it all to the prepared namespace or the metaclass, I
    now look for a special method in the Enum class, and if present use
    that
- removed settings (no AutoNumber, AutoName, nor Unique)
- redid AutoEnum to use the _generate_next_value_ technique
- updated docs

Let me know what you think!
msg271855 - (view) Author: John Hagen (John Hagen) * Date: 2016-08-02 21:31
@Ethan

I reviewed your latest patch.  I think it's a good step forward in terms of simplicity.  Most of my comments were not major.

Even before this patch, I was mulling around how enum.Unique, enum.UniqueEnum, and enum.unique seemed to violate the "There should be one obvious way to do it" principle, so I like that you omitted those in the latest patch.

Looks good to me, thanks for all of your work!
msg271868 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-03 01:55
Okay, I think I'm done making changes unless any more mistakes are found.

Thanks everyone for the pushing, prodding, and feedback!
msg272071 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-08-05 23:18
New changeset 7ed7d7f58fcd by Ethan Furman in branch 'default':
Add AutoEnum: automatically provides next value if missing.  Issue 26988.
https://hg.python.org/cpython/rev/7ed7d7f58fcd
msg272874 - (view) Author: John Hagen (John Hagen) * Date: 2016-08-16 18:58
I think there is a small typo in the Changelog / What's New.  The Issue isn't hyper-linked:

https://docs.python.org/3.6/whatsnew/changelog.html#id2
msg272876 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-16 19:30
added missing '#'
msg272885 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-16 21:46
FYI Raymond Hettinger started a discussion on Twitter about this feature, and the feedback may not be as good as you expected:
https://twitter.com/raymondh/status/765652816849285120

(I dislike this new magic thing, but I also never used the enum module, so I'm not sure that my opinion matters ;-))
msg272901 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-08-17 04:52
The use of a bare identifier as a self-assigning statement is unprecedented in the world of Python.  PyFlakes flags it as an error (because prior to now, a bare identifier in a class statement was almost always an error).  I suspect this will cause issues with other static analysis and refactoring tools as well:

    --- tmp_enum_example.py -----
    from enum import AutoEnum

    class Color(AutoEnum):
        red
        green = 5
        blue

    print(list(Color))

    --- bash session ------------
    $ py -m pyflakes tmp_enum_example.py
    tmp_enum_example.py:4: undefined name 'red'
    tmp_enum_example.py:6: undefined name 'blue'
    tmp_enum_example.py:11: undefined name 'yellow'


Also, the usual technique of commenting out blocks with triple quotes introduces unexpected breakage:

    --- tmp_enum_example2.py -----
    from enum import AutoEnum

    class Color(AutoEnum):
        red
        green = 5
        blue
        ''' XXX temporarily comment these out
        brown
        orange
        '''
        yellow

    print(list(Color))

    --- bash session ------------
    $ py -m tmp_enum_example.py
    [<Color.red: 1>, <Color.green: 5>, <Color.blue: 6>, <Color.yellow: 7>]
    /Users/raymond/cpython/python.exe: Error while finding spec for
    'tmp_enum_example.py' (AttributeError: module 'tmp_enum_example'
    has no attribute '__path__')

I worry that this proposal is worse than just being non-idiomatic Python.  In a way, it undermines pillars of the language and conflict everyone's mental model of how the language works.  Historically, a bare variable name raised a NameError if undefined and would otherwise act as a expression who's result was discarded.  However, as used here, it fiats an attribute into existence and assigns it a value.  That to my eyes looks like a new language that isn't Python.  This is really weird and undermines my expectations.

The need for the "ignore" parameter for the "shielded set" is a hint that the patch is working against the grain of the language and is not in harmony with Python as a coherent whole.  It is a harbinger of problems to come.

Lastly, I question whether there is any real problem being solved.  You already have "Color = IntEnum('Animal', 'red green blue')" that works well enough, doesn't mess with language norms, that works nicely with triple quotes for multiline entries, and that extends easily to hundreds of constants.

It seems to me that too much magic and unidiomatic weirdness are being leveled at too small of a problem.  Plus we already have one way to do it.

In teaching people to make effective use of the language, a key learning point is learning the portents of trouble to come and recognizing that that not everything that can be made to work should actually be done.

Please reconsider whether you really want to open this Pandora's box.  Right now, it's not too late.  No doubt that you will find some people who like this (it reminds them of C), but you will also find some very experienced developers who are made queasy by the bare identifier transforming from an expression into an assigning statement.  This more than an "if you don't like it, don't use it" decision, I think an important and invisible line is being crossed that we will regret.

P.S.  A lesson I learned from maintaining the itertools module is that adding more variants of a single idea tends to make the overall toolkit harder to learn and impairs usability.  Users suffer when given too many choices for closely related tasks.  The "one way to do it" line in the Zen of Python is there for a reason.
msg272932 - (view) Author: John Hagen (John Hagen) * Date: 2016-08-17 12:56
@Raymond, you raise valid concerns to be sure.  Hoping we can work something out.

@Ethan, what are your thoughts?


It's not just C that has enums where you can define a unique group of names and omit the values for clarity when they are not significant:  

C++: http://en.cppreference.com/w/cpp/language/enum
C#: https://msdn.microsoft.com/en-us/library/sbbt4032.aspx
Java: https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html
Rust: https://doc.rust-lang.org/book/enums.html

In my experience this is the most common and simple use case for enums.

Raymond, what are your thoughts about the version of AutoEnum that requires that a bare tuple be used as the value.  It has been in the Python docs since 3.4 and was actually the original request of this issue: https://docs.python.org/library/enum.html#autonumber

It avoids many of the concerns that you've raised while still providing a way to create Enums in the normal class declaration method users would expect with a minimum amount of boilerplate.  Note that normally you want to use @enum.unique with a normal Enum, but AutoEnum also allows you to omit that boilerplate as you can't accidentally alias the values.

@enum.unique
class Color(enum.Enum):
    aquamarine = 1
    blue = 2
    fushia = 3
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in having to increment all following members
    ...
    green = 40
    red = 41

vs.

class Color(enum.AutoEnum):
    aquamarine = ()
    blue = ()
    fushia = ()
    # inserting a member here (perhaps because it's clearest to keep these in alphabetic order)
    # results in no refactoring
    ... (30+ more)
    green = ()
    red = ()

A big advantage of the class based Enums compared to the functional API is that you can clearly document an Enum and its members in way Sphinx can take advantage of and developers would be used to.


# Assuming tuple assignment version for this example.
class ClientOperationMode(enum.AutoEnum):
    """Modes of operations of the network client."""

    push = ()
    """The client pushes data to the server automatically."""

    pull = ()
    """The client pulls commands from the server."""

    hybrid = ()
    """The client both pushes data and pulls for commands from the server."""

Sphinx will document this AutoEnum like a normal class, pulling in the class docstring, and attribute docstrings.

I don't see an obvious way to do this in the functional API docs: https://docs.python.org/library/enum.html#functional-api
msg272941 - (view) Author: Anilyka Barry (abarry) * (Python triager) Date: 2016-08-17 13:37
I tend to like all things magic, but the more I think about it, and the less I like it being a part of the standard library. I've had a use for this feature before, and when I did, I cooked my own 12-lines subclass of EnumMeta and _EnumDict. Raymond's points are pretty spot-on, and I also think that this shouldn't go in the stdlib. There's still time.

While this particular flavour of magic sounds too, well, magic, for Python or even myself, I think a less magical approach can be taken. Namely, something like AutoNumberedEnum which requires values to be empty tuples is being explicit about what you want and not allowing typos in the class definition. Raymond's point about (temporarily) commenting out enum members breaking the order highlights this, and while this approach doesn't solve it, it makes it obvious that there is *something* that changed.

Another approach, which doesn't exclude the above, is to make EnumMeta and _EnumDict public and documented classes (!), thus allowing people like me to create their own blueberry-flavoured magic enumerations without any impact on the people who don't use my code. Or if I don't feel like reinventing the wheel, I can just pip install the module and use that instead.

I think that the whole idea of making enums in Python work like they do in C is looking at the problem from the wrong angle. It's true that Python takes some of its design from C, but naively trying to replicate C-like behaviour with C-like syntax doesn't work all the time. How often do beginners think 'x ^ y' means 'x to the power of y', and are then surprised by the behaviour?

I think this version of Enum raises the barrier to entry for new programmers. Enum is a nice feature, and it helps new and old programmers alike write clean(er) code for various purposes. When a programmer sees this use, they won't think "oh this must call __getitem__ and then assign an automatic value", they'll wonder "why is this code even running?". And then it's up to the Raymonds of this world to explain that Enum uses a metaclass (which, I'm sure, is not a topic they'll want to tackle by the time these programmers reach Enum) and that the mapping overloads __getitem__.

All in all, this magical approach is just too magical for Python. I understand that magic is fun to have and play with, but the standard libary isn't where you should keep your toys. I use a throwaway repo for all my magic this-is-not-a-good-idea-but-I-wanna-do-it-anyway ideas, and this is where I think such magic goes. It definitely doesn't belong in the standard library, within an arm's reach of the first curious programmer to wander there.
msg272951 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-17 14:24
> It's not just C that has enums where you can define a unique group of names and omit the values ...

Yes, Python 3.4 too: Animal = Enum('Animal', 'ant bee cat dog')

https://docs.python.org/dev/library/enum.html#functional-api
msg272952 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-17 14:25
> Raymond, what are your thoughts about the version of AutoEnum that requires that a bare tuple be used as the value.  It has been in the Python docs since 3.4 and was actually the original request of this issue: https://docs.python.org/library/enum.html#autonumber

Well, I suggest to keep it as a recipe in the doc ;-)
msg272971 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-17 17:19
Raymond, I appreciate your review and your poll.  I am open to removing AutoEnum, but would like to give it a couple more weeks of review.  (I'll post on py-dev.)

The only point you made that I will firmly refute is the "unexpected breakage": you ran your test script with the -m "run module as script" flag, which is what caused the problem.
msg272982 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-08-17 18:30
> you ran your test script with the -m "run module as script" flag

Right, that was a mistest, so it looks like triple quotes do work.

I did notice that there's also an issue if one line reads, "red='hello'".

But really, the big issue is using a bare-identifier to fiat an attribute into existence.  That's a door that really shouldn't be opened.

Secondarily, the doesn't seem to be any use case that can't be readily covered by the existing classes.  There is no real need for the witchcraft and the departure from Python norms.
msg273006 - (view) Author: David Hagen (David Hagen) Date: 2016-08-18 00:52
> Secondarily, the doesn't seem to be any use case that can't be readily covered by the existing classes.

The use case that doesn't have a clean interface in 3.5 at the moment is the most common use case of enums: just a collection of named objects of given type; I don't care about the values because they don't really have values apart from their object identities.

When writing enums in Rust, Swift, C#, etc., the bare identifier not only saves typing--it allows the developer to explicitly indicate that the underlying value has no meaning. (It may be better to use "object()" rather than an integer on AutoEnum, but that is not very important.)

It was said that Python has this feature already:

> Yes, Python 3.4 too: Animal = Enum('Animal', 'ant bee cat dog')

I will concede that this can do what I want. I hope others will concede that this is not a clean interface. The class name is duplicated and the members are regexed out of a space-delimited string. This same argument could be made to deprecate the unnecessary "class" keyword in favor of the "type" function.

I will also concede that there is some deep magic going on in AutoEnum and that magic should be avoided when it obscures. I personally think the only people who will be truly surprised will be those who already know Python at a deep enough level to know that deep magic must be required here. Everyone else will see "Enum" and a list of bare identifiers, and correctly conclude that this is your basic enum from everyone other language.

Perhaps an ideal solution would be an enum keyword:

enum PrimaryColor:
    red
    blue
    green

But that's not happening ever.

The next best solution is the current implementation:

class PrimaryColor(AutoEnum):
    red
    blue
    green

But because of the magic, it only barely beats out what I think is the other great solution already mentioned here:

class PrimaryColor(AutoEnum):
    red = ()
    blue = ()
    green = ()

These two solutions are isomorphic. Both save the developer from having to provide a (possibly meaningless) value. Both let docstrings be added. Both provide the ability to reorganize without renumbering. The last one trades magic for boilerplate.

I'll keep using them from the aenum package if they don't make it into 3.6, but I think this is a fundamental enough construct that it belongs in the standard library. It is hard to convince tool maintainers to fully support these until they are blessed here.
msg273039 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2016-08-18 15:23
Temporarily, marking this as open so that more people can see the new comments.

For John, I'm not sure what I can say that will help you.  The goal of the standard libraries modules is to provide tools the use language we have, not to invent a new language the isn't really Python or Pythonic.  In the absence of an "enum" keyword, you have a number of ways to work within the language.

In this case (wanting auto numbering but not caring what the values are), you already have several ways to do it.  I your issue is not where the use case is met; instead, you just don't like how their spelled (because it isn't exactly how it it looks in C).

This already works:  Animal = Enum('Animal', ['ant', 'bee', 'cat', 'dog']).  This is very flexible and lets you read the constants from many possible sources.

If you're attracted to multiline input, that is already possible as well:

    Animal = Enum('Animal', '''
	ant
	bee
	cat
	dog
    ''')

It is clear that you're bugged by writing Animal twice, but that is how Python normally works and it is a very minor nuisance (it only occurs once when the enum is defined).  Note, you already are rewriting "Animal" every time you use the enum value (presumably many times):  board_ark(Animal.ant, Animal.bee)

This whole feature request boils down to wanting a currently existing feature to be spelled a little differently, in a way that doesn't look like normal Python.  Elsewhere, we resisted the temptation to alter the language look-and-feel to accommodate small spelling tweaks for various modules (i.e. we pass in normal strings to the regex module even though that sometimes requires double escaping, we pass in the class name to namedtuples even though that uses the class name twice, the xpath notation in passed into XML tools as strings even though parts of it look like regular langauge, we don't abuse the | operator to link together chains of itertools, etc)

Since the enum module already provides one way to do it, I recommend that we stick with that one way.  Creating too many variants does not help users.  Using the core language in odd ways also does not help users.
msg273040 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-08-18 15:33
Ethan, the suggestion has come up several times about using a dummy value such as the empty tuple to do autonumbering, thus looking more Pythonic.  I'm not a huge fan of the empty tuple, and I'm still not sure whether we need this, but I wonder if it would be possible to not have a new base class, but to put the smarts in the value to which the enums were assigned.  E.g. is this possible (a separate question than whether it's good :):

from enum import Enum, auto

class Color(Enum):
    red = auto
    green = auto
    blue = auto

Apologies if this has already been suggested; this tracker thread is too long to read the whole thing. :(
msg273064 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-18 19:49
Thank you, Raymond, David, Barry, John, etc. for your feedback.

While I really like AutoEnum I can see that that much magic doesn't need to exist in the stdlib.

Unfortunately, the alternatives aren't very pretty, so I'll leave the AutoNumberEnum as a recipe in the docs, and not worry about an 'auto' special value.

For those that love AutoEnum, it will be in the aenum third-party package.
msg273065 - (view) Author: John Hagen (John Hagen) * Date: 2016-08-18 19:57
Ethan, thank you so much for all of your work.  Looking forward to any future collaboration.
msg273076 - (view) Author: Vedran Čačić (veky) * Date: 2016-08-19 04:47
I don't think the problem is with (Auto)Enum at all. Problem is in a different place, and this is just a symptom.

Problem is that people make too many assumptions about `class` suite. Yes, type is default metaclass. But not every class should be constructed by type's rules - Python provides a very detailed and powerful mechanism for specifying a different set of rules, should you need it. This is what metaclass keyword argument should be all about. And that is the true reason why metaclass conflict is an issue: every metaclass is a DSL specification, and the same suite cannot be interpreted according to two different DSLs at the same time.

But since there's a widespread myth that users don't like to type, implementors use a trick, "class Spam(metaclass=SpamMeta): pass", and say to their users: "there, just inherit from Spam like you inherit from ordinary classes". That way, we spare them a few keystrokes, sparing them also of opportunity to learn that metaclasses _do_ change the semantics of what they see indented after the colon.

I wonder what Raymond's poll result would be if a normal way to write such code would expose the metaclass

    class Color(metaclass=AutoEnum):
        blue
        yellow

? Raymond has many valid objections, but they all apply to "ordinary" classes, instances of type. Color is not an instance of type, or at least conceptually it isn't. type (as a metaclass in Python) means a blueprint, a way to construct new instances via __call__, and every instance of it has that behavior, some of which even customize it by defining __new__ and/or __init__. type is also special because its power of creation is transitive: its instances know how to produce their instances in almost the same way it does.

But nothing of it is mandatory, and in fact it just stands in the way when we try to define Enum (or singletons, or database models, or... whatever that is not really a Python type). We do inherit from type because it's easier, and then usurp and override its __call__ behavior to do something almost entirely different. (Don't you feel something is wrong when your __new__ method doesn't construct a new object at all?:) It's completely possible, but it's not the only way.

Maybe class is just too tainted a keyword. There was a thread on python-ideas, that we should have a new keyword, "make". "make m n(...)" would be a clearer way to write what's currently written "class n(..., metaclass=m)", with a much more prominent position for the metaclass, obviating the need for "just inherit from Spam" trick, and dissociating in people's minds the connections of arbitrary metaobjects with type. ("class" keyword could be just a shortcut for "make type" in that syntax.) But of course, a new keyword is probably too much to ask. (A mild effect can be gained by reusing keywords "class" and "from", but it looks much less nice.)

However, there is _another_ underused way metaclasses can communicate with the external world of users of their instances: keyword arguments. I wonder if Raymond's objections would be as strong if the autovivification was explicitly documented?

    class Color(Enum, style='declarative'):
        blue
        yellow

Of course we can bikeshed about exact keyword argument names and values, but my point is that there is an unused communication channel for converting typical Python user's surprise "why does this code even work" into "hey, what does that style='declarative' mean?"
msg273084 - (view) Author: Kenneth Reitz (kennethreitz) Date: 2016-08-19 07:03
This addition to Python, specifically the bare assignment-free `red` syntax would be a huge mistake for the language. It looks like a syntax error to me, and completely destroys my mental model for how Python code either does work or should work. 

A simple assignment to `None`, `()`, or imported static `AUTO_ENUM` (or similar, akin to subprocess.PIPE) would be *much* preferred.
msg273092 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-19 08:20
Would it be possible to use a auto or AUTO_ENUM constant from the enum in
the Enum class? (And drop AutoEnum class.)
msg273103 - (view) Author: Vedran Čačić (veky) * Date: 2016-08-19 09:12
I must say I never understood how exactly is assigning a constant better in this regard. To me, after

    a = SOMETHING
    b = SOMETHING

a and b should be the same, right? _Especially_ when SOMETHING is None or (), but also when it's some bare identifier like AUTO_ENUM.

Mental models are already broken. We are already "lying" according to semantics of type metaclass. There is no reason then to have to sprinkle some magic dust over our code, just so it looks like an assignment is the most important thing here, when it really isn't.

Either we should embrace Python's power in full and communicate to users that there's something weird going on, or we shouldn't have AutoEnum. To me, `green = None` is in no way better, in fact it seems more dishonest, than simply `green`. Again, _if_ we communicate (e.g. via style='declarative') that the semantics is not ordinary. (Alternatively, we might just write `green = uuid4()` and be done with it.:)
msg273105 - (view) Author: Kenneth Reitz (kennethreitz) Date: 2016-08-19 11:48
Explicit is better than implicit :)
msg273109 - (view) Author: Vedran Čačić (veky) * Date: 2016-08-19 12:19
Absolutely. How the hell is `green = None` explicit?!

On the other hand, `class Color(Enum, style='declarative'):` is explicit. People must learn something. Why then don't they learn the right thing instead of "hey, assigning None inside enums magically does the right thing" - except when it doesn't.

Just a hint of a nightmare scenario: you write a method decorator, but you forget to return a value from it (happened to me more times than I care to admit). Ooops, your method is now next member of your enum. Good luck debugging that. :-O

In fact, what _is_ explicit, is this:

    class Color(metaclass=Enum):
        green = object()
        yellow = object()

I could live with it. :-)
msg273118 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-19 13:29
2016-08-19 14:19 GMT+02:00 Vedran Čačić <report@bugs.python.org>:
> In fact, what _is_ explicit, is this:
>
>     class Color(metaclass=Enum):
>         green = object()
>         yellow = object()
>
> I could live with it. :-)

For me, it's legit to use a singleton in an enum. I mean that I expect
that green != yellow. Some remark for an empty tuple.

So I woud really prefer a contant from enum like AUTO_ENUM.
msg273125 - (view) Author: Vedran Čačić (veky) * Date: 2016-08-19 14:34
> For me, it's legit to use a singleton in an enum.

But you cannot explain _why_, right? Like Raymond says, it's not working like anything else in Python works.

To me, this looks almost the same as the gripes of those lazy people who want "myfile.close" expression to close myfile, or worse, "quit" to exit the Python interpreter. An irrational phobia of parentheses. Why not call your object, if you expect some code to be executed? [At least a dotted name like myfile.close _could_ be made to work using descriptors, but still I think everyone agrees it's not a good idea.]

To me, we either want to stay with default type metaclass (`style='imperative'`:), and write something like

    class Color(Enum):
        green = member()
        yellow = member()

or we acknowledge that `style='declarative'` has its place under the sun (not only for Enums, of course... database models, namedtuples, even ABCs could profit from it), and embrace its full power, realizing it's not Python we usually see, but it's still Python.

Magic is not something to be afraid of, if you understand it. We are talking Py3.6 here... aren't formatted strings a totally insane magic? Yet they ended up in the language, because they are immensely better than the alternatives. Here the gain is much smaller, but the threshold is much lower too: we don't need new syntax at all.
msg273126 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-19 14:41
Vedran, you have some very interesting points.  However, if you really want to champion this you'll need to take it to Python Ideas.
msg273127 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2016-08-19 14:45
Hey, I just realized that you can get pretty darn close with just a little bit
of extra typing, and existing stdlib:

    from enum import Enum
    from itertools import count

    auto = count()

    class Color(Enum):
        red = next(auto)
        green = next(auto)
        blue = next(auto)

Look ma, no magic!
msg273131 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-08-19 14:53
No magic, but a whole heap of extra boiler-plate.  :(
msg273134 - (view) Author: Kenneth Reitz (kennethreitz) Date: 2016-08-19 15:11
There's a difference between boiler-plate and "code".

> On Aug 19, 2016, at 10:53 AM, Ethan Furman <report@bugs.python.org> wrote:
> 
> 
> Ethan Furman added the comment:
> 
> No magic, but a whole heap of extra boiler-plate.  :(
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26988>
> _______________________________________
msg273145 - (view) Author: David Hagen (David Hagen) Date: 2016-08-19 17:11
One solution similar to one proposed by Vedran works with the current Enum:

    class Color(Enum):
        red = object()
        green = object()
        blue= object()

I tested this in PyCharm and it is perfectly happy with the autocomplete and everything. The main disadvantage is the boilerplate, of course. And perhaps "object()" does not show the clearest intent, but it depends on your perspective. The repr also looks kind of funny:

    >>>  repr(Color.red)
    <Color.red: <object object at 0x7fb2f353a0d0>>

One possibility would be to add an auto() function to enum as a wrapper around object(), providing a more explicit name and a cleaner repr:

    from enum import Enum, auto

    class Color(Enum):
        red = auto()
        blue = auto()
        green = auto()

    repr(Color.red)
    <Color.red>
    # auto() means it has no (meaningful) value, so show nothing
msg273154 - (view) Author: Vedran Čačić (veky) * Date: 2016-08-19 19:33
# Just wait until I put the keys to the time machine in their usual place... :-)

Ok, while we're talking about whether declarative style is a good idea, Python has already got the initial seed of that style. With Guido's blessing! PEP 526 enables us to mix declarative and imperative style in "ordinary" code, so we can write

    @Enum
    class Color:
        green: member
        yellow: member

without any additional syntax. I think it satisfies everyone: there are no parentheses, and there are no assignments. [_And_ there is no misleading analogy with existing syntax, because this is a new syntax.:] There are just declarations, and the decorator instantiates them.

Decorator is needed because formally we need to exclude the type checking semantics, and the only official way currently is through a decorator. But in fact we can use the forward references to _actually_ annotate the members with their real type:

    class Color(Enum):
        green: 'Color'
        yellow: 'Color'

And once the forward references get a nicer syntax, and the unpacking issues are solved, we'll be able to write

    class Color(Enum):
        green, yellow: Color

And I think finally everyone will be happy. :-)
msg273188 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-08-20 07:01
New changeset 2e243f78720e by Ethan Furman in branch 'default':
Issue26988: remove AutoEnum
https://hg.python.org/cpython/rev/2e243f78720e
History
Date User Action Args
2022-04-11 14:58:30adminsetgithub: 71175
2016-08-20 07:02:02ethan.furmansetstatus: open -> closed
stage: resolved
2016-08-20 07:01:19python-devsetmessages: + msg273188
2016-08-19 19:33:58vekysetmessages: + msg273154
2016-08-19 17:11:36David Hagensetmessages: + msg273145
2016-08-19 15:11:24kennethreitzsetmessages: + msg273134
2016-08-19 14:53:31ethan.furmansetmessages: + msg273131
2016-08-19 14:45:17barrysetmessages: + msg273127
2016-08-19 14:41:48ethan.furmansetmessages: + msg273126
2016-08-19 14:34:13vekysetmessages: + msg273125
2016-08-19 13:29:23vstinnersetmessages: + msg273118
2016-08-19 12:19:30vekysetmessages: + msg273109
2016-08-19 11:48:00kennethreitzsetmessages: + msg273105
2016-08-19 09:12:17vekysetmessages: + msg273103
2016-08-19 08:20:46vstinnersetmessages: + msg273092
2016-08-19 07:03:21kennethreitzsetnosy: + kennethreitz
messages: + msg273084
2016-08-19 04:47:25vekysetnosy: + veky
messages: + msg273076
2016-08-18 19:57:11John Hagensetmessages: + msg273065
2016-08-18 19:49:54ethan.furmansetresolution: fixed -> rejected
messages: + msg273064
stage: resolved -> (no value)
2016-08-18 15:33:47barrysetmessages: + msg273040
2016-08-18 15:23:53rhettingersetstatus: closed -> open

messages: + msg273039
2016-08-18 00:53:00David Hagensetnosy: + David Hagen
messages: + msg273006
2016-08-17 18:30:28rhettingersetmessages: + msg272982
2016-08-17 17:19:16ethan.furmansetmessages: + msg272971
2016-08-17 14:25:36vstinnersetmessages: + msg272952
2016-08-17 14:24:12vstinnersetmessages: + msg272951
2016-08-17 13:37:08abarrysetnosy: + abarry
messages: + msg272941
2016-08-17 12:56:39John Hagensetmessages: + msg272932
2016-08-17 04:52:02rhettingersetnosy: + rhettinger
messages: + msg272901
2016-08-16 21:46:44vstinnersetnosy: + vstinner
messages: + msg272885
2016-08-16 19:30:30ethan.furmansetmessages: + msg272876
2016-08-16 18:58:14John Hagensetmessages: + msg272874
2016-08-09 01:08:28ethan.furmansetstatus: open -> closed
resolution: fixed
stage: resolved
2016-08-05 23:24:36ethan.furmanlinkissue26981 superseder
2016-08-05 23:18:44python-devsetnosy: + python-dev
messages: + msg272071
2016-08-03 01:55:43ethan.furmansetfiles: + issue26988.stoneleaf.05.patch

messages: + msg271868
2016-08-02 21:31:29John Hagensetmessages: + msg271855
2016-08-02 07:27:31ethan.furmansetfiles: + issue26988.stoneleaf.03.patch

messages: + msg271811
2016-07-16 00:28:10ethan.furmansetmessages: + msg270524
2016-07-16 00:25:42ethan.furmansetfiles: + issue26988.stoneleaf.02.patch
assignee: ethan.furman
messages: + msg270523
2016-07-12 11:52:31John Hagensetmessages: + msg270238
2016-07-12 01:45:10ethan.furmansetfiles: + issue26988.stoneleaf.01.patch
keywords: + patch
messages: + msg270219
2016-07-11 22:47:23John Hagensetmessages: + msg270212
2016-07-11 19:25:50barrysetmessages: + msg270201
2016-07-11 19:05:27ethan.furmansetmessages: + msg270199
2016-07-11 01:34:47ethan.furmansetmessages: + msg270151
2016-07-11 01:25:38barrysetmessages: + msg270148
2016-07-11 00:27:17ethan.furmansetmessages: + msg270142
2016-07-10 22:26:04John Hagensetmessages: + msg270137
2016-07-10 18:09:04barrysetmessages: + msg270117
2016-07-10 17:42:18ethan.furmansetmessages: + msg270107
2016-07-10 14:35:34John Hagensetmessages: + msg270088
2016-07-10 05:41:37ethan.furmansetmessages: + msg270071
2016-07-09 17:11:57ethan.furmansetmessages: + msg270056
2016-07-09 17:02:14John Hagensetmessages: + msg270054
2016-07-09 16:19:28ethan.furmansetmessages: + msg270051
2016-07-09 16:04:43barrysetmessages: + msg270049
2016-07-09 15:20:16ethan.furmansetmessages: + msg270048
2016-07-09 13:04:55John Hagensetmessages: + msg270039
2016-06-29 15:22:04barrysetmessages: + msg269521
2016-06-29 11:33:17John Hagensetmessages: + msg269476
2016-05-10 11:17:13John Hagensetmessages: + msg265237
2016-05-10 02:06:34ethan.furmansetmessages: + msg265218
2016-05-10 02:01:21barrysetmessages: + msg265216
2016-05-10 01:58:14ethan.furmansetmessages: + msg265215
2016-05-10 01:53:58ethan.furmansetnosy: + barry, eli.bendersky
2016-05-10 01:34:31John Hagencreate