Title: Implement PEP 422: Simple class initialisation hook
Type: enhancement Stage: resolved
Components: Interpreter Core, Library (Lib) Versions: Python 3.5
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, cheryl.sabella, daniel.urban, eric.snow, ezio.melotti, flox, jcea, ncoghlan
Priority: normal Keywords: needs review, patch

Created on 2013-01-26 21:29 by daniel.urban, last changed 2018-01-28 05:09 by ncoghlan. This issue is now closed.

File name Uploaded Description Edit
pep422_1.patch daniel.urban, 2013-01-26 21:29 PEP 422 implementation (1) review
pep422_2.patch daniel.urban, 2013-01-27 07:15 PEP 422 implementation (2) review
pep422_3.patch daniel.urban, 2013-01-30 20:14 PEP 422 implementation (3) review
pep422_4.patch daniel.urban, 2013-02-17 10:58 PEP 422 implementation (4) (object.__init_class__) review
pep422_5.patch daniel.urban, 2013-02-17 15:45 PEP 422 implementation (5) (document object.__init_class__) review
pep422.patch daniel.urban, 2013-02-17 15:46 Patch to the PEP 422 itself
pep422_6.patch daniel.urban, 2013-04-14 13:59 PEP 422 implementation (6) (namespace argument of __prepare__ and __init_class__) review
Messages (18)
msg180714 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-01-26 21:29
The attached patch implements PEP 422 -- Simple class initialisation hook (__init_class__). It includes tests, but it doesn't include documentation yet.
msg180746 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-01-27 07:15
Thanks for the review Ezio!

I've fixed the order of the arguments of assertEqual. Regarding your other comment: I agree that your code is shorter and clearer, but it doesn't do exactly the same thing. cls.__init_class__ can be None, and in that case, we should call it (the user should get a TypeError). So the getattr call would need another sentinel object. And with that sentinel, I don't think the code would be shorter and clearer any more.
msg180990 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-01-30 20:14
I'm attaching a new patch with some documentation and one additional test.
msg181795 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-02-10 12:21
I've belatedly applied the PEP update Daniel sent me, and added a reference to this issue from the PEP.

The latest version of the patch looks very good to me, just one very minor nit with the phrasing in the docs. Specifically, it is better to replace "like" with "as" in the phrase "like in the following example" to get "as in the following example".
msg182265 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-02-17 10:50
Thanks for the grammar correction, I've fixed it in the new patch.

The new patch also adds object.__init_class__ (which is a no-op), to support cooperative multiple inheritance of __init_class__. (Adding type.__init_class__ was mentioned in the python-dev discussion, but that is not sufficient, so I've added this method to object instead of type.)

Tests are also updated. I'll update the docs, if this change is OK.
msg182267 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-02-17 11:50
Ah, I think I see your point - because __init_class__ is supposed to be a class method on instances of the metaclass, the anchor needs to be on object (the highest level instance of the default metaclass), not on type if we don't want super to behave strangely?

I think that's a valid conclusion (albeit a very subtle difference relative to an ordinary method on type), and compared to the tapdancing we do in __new__ and __init__ to help them be valid anchors for cooperative multiple inheritance, adding object.__init_class__ is a relatively minor thing.

If you wanted to propose a patch to the PEP as well, that would be great.
msg182275 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-02-17 15:45
Yes, if we would add a regular (instance) method __init_class__ to type, it could (probably) work for regular (non-meta) classes, but not for metaclasses.  If a metaclass Meta wouldn't define __init_class__ itself, calling Meta.__init_class__() in __build_class__ wouldn't work, since Meta would inherit the *instance* method from type (its superclass).  We might be able to make this work somehow (I'm not sure), but I think adding it to object as a classmethod works fine, and doesn't require special casing.

The attached patch documents, that object has an __init_class__ (and also adds some extra tests). I'll attach a patch to the PEP as well.
msg184092 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-03-13 17:08
I've looked into implementing the changes in the new version of the PEP. It seems, that currently type.__new__ copies the dict returned by __prepare__ ( I think this means that some of the new examples wouldn't work ("Order preserving classes" and "Extending a class"). It also means, that currently it is impossible for two classes to share a namespace object (possibly this was the intention of copying the dict). So the "Note" in the PEP wouldn't be necessary (unless type.__new__ were modified).

I'm not sure if the PEP proposes to change this behavior of type.__new__ or not...
msg184138 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-03-14 07:24
I hadn't noticed that type.__new__ copied the contents (it surprises me that it does both that *and* restricts the input type to a true dict instance).

The "Extending a class" example should still work as shown, since the magic of that happens while the body of ExtendedExample is running.

For the order preserving case, it turns out CPython already keeps a copy of the original namespace around as cls.__locals__, but this is currently undocumented (as far as I can tell anyway).

If we elevate that to documented behaviour, then __init_class__ implementations can reference both the original object, as well as the snapshot underlying the class object.

Given that, it is probably also better to revert the namespace keyword to accepting an instance rather than a factory function, since the copy operation after execution of the class body is automatic.
msg184189 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-03-14 20:42
We should definitely have a way to expose the original dictionary from __prepare__().  Along with Nick's point, another reason is to allow class decorators to have access to that original, which is important to any use case that involves post-creation introspection of the definition order within the class namespace.

Nick, where did you see cls.__locals__?  I'm not finding any mention of __locals__ outside compiler.c and symtable.c.  I agree that such an attribute on classes would be helpful, even if by another name, and that it should be documented.
msg184190 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-03-14 20:44
I've also opened #17421 for dropping the restriction on the namespace passed to the metaclass and #17422 for documenting that the passed namespace is copied into a new dict.
msg184192 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-03-14 21:04
> Given that, it is probably also better to revert the namespace keyword
> to accepting an instance rather than a factory function, since the copy
> operation after execution of the class body is automatic.

Agreed.  Of course, the related note is rendered superfluous.  There is still the possibility of reusing the same namespace across successive class definitions, with the associated consequences.  However, since type.__new__() copies the namespace into a new dict, it mitigates the main concern: implicit modification to an existing class in the definition of another.
msg184195 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-03-14 21:34
Oh, that's bizarre - the presence of __locals__ is a side effect of
calling locals() in the class body. So perhaps passing the namespace
as a separate __init_class__ parameter is a better option.
msg184310 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-03-16 13:16
Eric and I discussed this further this morning. We were interested in two main points:
1. When no custom namespace is used, we definitely shouldn't double the storage requirements for the class object
2. If a custom namespace is used, it would be nice to make it available to both the __init_class__ hook *and* to class decorators

The design we came up with for doing so is for type.__new__ to include the equivalent of:

    if type(namespace) != dict:
        cls.__namespace__ = types.MappingProxyType(namespace)

The practical downside of that approach is that it still doubles the storage requirements for every class that uses a custom namespace, even if the custom namespace isn't actually needed after creation. The upside is that you can write a class decorator that needs ordering information and say "to use this decorator, you must also specify 'namespace=collections.OrderedDict()'"

There's also a major complexity downside to that approach - the distinction between __dict__ and __namespace__ is somewhat subtle, especially since __namespace__ won't see later changes made through setattr() and delattr(), and __dict__ won't see changes made through any external references to the custom namespace.

That means my current inclination is to instead change the signature of __init_class__ to accept a read-only view of the execution namespace as a second parameter. Decorators won't have access to the details lost by the copying of attributes into an ordinary dict unless used with a metaclass or base class that takes steps to attach it to the created class object. I can live with that limitation, and it means we only have to properly document the "the contents of the class execution namespace are copied into an ordinary dict instance when the class is created" behaviour of type to explain why __init_class__ has the second parameter.
msg184322 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2013-03-16 15:12
I'm fine with that.  Would it make sense to have the signature of __init_class__ parallel meta.__init__():

  __init_class__(cls, name, bases, namespace)

or even

  __init_class__(cls, name, bases, ns, *, namespace=None, **kwds)
msg186920 - (view) Author: Daniel Urban (daniel.urban) * (Python triager) Date: 2013-04-14 13:59
I've attached a new patch. With this patch, type.__prepare__ has an optional keyword-only argument 'namespace', and returns it if it's specified. Also, __init_class__ is passed an argument: a mapping proxy of the mapping originally returned by __prepare__.

> Would it make sense to have the signature of __init_class__ parallel
> meta.__init__()
I don't think so, because some of the arguments (name, bases) would be mostly useless, others would have a different meaning (namespace). Although, passing the keyword arguments from the class header might make some sense ... I'm not sure.

If everybody agrees with these changes, I'll create a patch for the PEP too.
msg310884 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2018-01-27 19:49
Since PEP 422 was withdrawn in favor of PEP 487, should this issue be closed?
msg310919 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-01-28 05:09
Indeed it should! Thanks for pointing that out :)
Date User Action Args
2018-01-28 05:09:30ncoghlansetstatus: open -> closed
resolution: out of date
messages: + msg310919

stage: patch review -> resolved
2018-01-27 19:49:51cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg310884
2015-07-21 08:01:18ethan.furmansetnosy: - ethan.furman
2014-05-19 01:42:43berker.peksagsetversions: + Python 3.5, - Python 3.4
2014-05-19 01:40:24ethan.furmansetnosy: + ethan.furman
2013-04-14 13:59:41daniel.urbansetfiles: + pep422_6.patch

messages: + msg186920
2013-03-16 15:12:45eric.snowsetmessages: + msg184322
2013-03-16 13:16:54ncoghlansetmessages: + msg184310
2013-03-14 21:34:50ncoghlansetmessages: + msg184195
2013-03-14 21:04:13eric.snowsetmessages: + msg184192
2013-03-14 20:44:49eric.snowsetmessages: + msg184190
2013-03-14 20:42:05eric.snowsetnosy: + eric.snow
messages: + msg184189
2013-03-14 07:24:32ncoghlansetmessages: + msg184138
2013-03-13 17:08:17daniel.urbansetmessages: + msg184092
2013-02-18 17:19:11jceasetnosy: + jcea
2013-02-17 20:59:31Arfreversetnosy: + Arfrever
2013-02-17 15:46:17daniel.urbansetfiles: + pep422.patch
2013-02-17 15:45:19daniel.urbansetfiles: + pep422_5.patch

messages: + msg182275
2013-02-17 11:50:24ncoghlansetmessages: + msg182267
2013-02-17 10:58:17daniel.urbansetfiles: + pep422_4.patch
2013-02-17 10:57:36daniel.urbansetfiles: - pep422_4.patch
2013-02-17 10:50:40daniel.urbansetfiles: + pep422_4.patch

messages: + msg182265
2013-02-11 11:37:11floxsetnosy: + flox
2013-02-10 12:21:59ncoghlansetmessages: + msg181795
2013-01-30 20:14:57daniel.urbansetfiles: + pep422_3.patch

messages: + msg180990
2013-01-27 07:15:59daniel.urbansetfiles: + pep422_2.patch

messages: + msg180746
2013-01-26 21:51:36ezio.melottisetnosy: + ezio.melotti
2013-01-26 21:29:18daniel.urbancreate