classification
Title: __init_subclass__ should be called in __init__
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.10
process
Status: closed Resolution: rejected
Dependencies: Superseder: [Enum] move member creation to __set_name__ in order to support __init_subclass__
View: 42901
Assigned To: ethan.furman Nosy List: carltongibson, ethan.furman, gvanrossum, ncoghlan, rhettinger, serhiy.storchaka, stutzbach
Priority: high Keywords:

Created on 2020-12-29 03:06 by ethan.furman, last changed 2021-01-26 15:07 by carltongibson. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23986 closed ethan.furman, 2020-12-29 05:28
Messages (7)
msg383946 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2020-12-29 03:06
PEP 487 introduced __init_subclass__ and __set_name__, and both of those were wins for the common cases of metaclass usage.

Unfortunately, the implementation of PEP 487 with regards to __init_subclass__ has made the writing of correct metaclasses significantly harder, if not impossible.

The cause is that when a metaclass calls type.__new__ to actually create the class, type.__new__ calls the __init_subclass__ methods of the new class' parents, passing it the newly created, but incomplete, class.  In code:

```
class Meta(type):
    #
    def __new__(mcls, name, bases, namespace, **kwds):
        # create new class, which will call __init_subclass__ and __set_name__
        new_class = type.__new__(mcls, name, bases, namespace, **kwds)
        # finish setting up class
        new_class.some_attr = 9
```

As you can deduce, when the parent __init_subclass__ is called with the new class, `some_attr` has not been added yet -- the new class is incomplete.

For Enum, this means that __init_subclass__ doesn't have access to the new Enum's members (they haven't beet added yet).

For ABC, this means that __init_subclass__ doesn't have access to __abstract_methods__ (it hasn't been created yet).

Because Enum is pure Python code I was able to work around it:
- remove new __init_subclass__ (if it exists)
- insert dummy class with a no-op __init_subclass__
- call type.__new__
- save any actual __init_subclass__
- add back any new __init_subclass__
- rewrite the new class' __bases__, removing the no-op dummy class
- finish creating the class
- call the parent __init_subclass__ with the now complete Enum class

I have not been able to work around the problem for ABC.

The solution would seem to be to move the calls to __init_subclass__ and __set_names__ to type.__init__.
msg383948 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2020-12-29 03:15
My understanding of using keywords in class headers

  class MyClass(SomeBaseClass, setting='maybe'):
    ...

is that the keywords would get passed into the super classes `__init_subclass__`  (`SomeBaseClass` and `setting`).

However, in the cases of 

- test_descr.py
- test_py_compile.py
- typing.py

that wasn't happening -- until the initial patch of moving the calls from `type.__new__` to `type.__init__`.

An `__init__` has been added in those three locations to discard the keyword arguments being passed in.
msg383960 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-12-29 05:34
Whoa, you start a discussion on python-dev and another on bpo? That sounds a bit hasty.
msg383962 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2020-12-29 06:24
Guido, I just wanted to get it all in place while it was fresh in my mind.  Actual code tends to make a discussion easier.  I'll make sure and transcribe any relevant discussion from python-dev to here.
msg384020 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2020-12-29 18:32
I see this as a backwards incompatible change and it's not worth doing for what seems to me a very minor benefit.
msg384876 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2021-01-12 01:29
Nick Coghlan made the observation that `__set_name__` should be doing what is currently the after-new work.

Tracking in #42901.
msg384877 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2021-01-12 01:32
Thanks, that's great! And thanks, Nick!
History
Date User Action Args
2021-02-01 20:09:42ethan.furmanunlinkissue35815 superseder
2021-01-26 15:07:32carltongibsonsetnosy: + carltongibson
2021-01-12 01:32:07gvanrossumsetmessages: + msg384877
2021-01-12 01:29:46ethan.furmansetstatus: open -> closed
superseder: [Enum] move member creation to __set_name__ in order to support __init_subclass__
messages: + msg384876

keywords: - patch
resolution: rejected
stage: patch review -> resolved
2020-12-29 18:32:24gvanrossumsetmessages: + msg384020
2020-12-29 06:24:20ethan.furmansetmessages: + msg383962
2020-12-29 05:34:10gvanrossumsetnosy: + gvanrossum
messages: + msg383960
2020-12-29 05:28:41ethan.furmansetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request22829
2020-12-29 05:26:33ethan.furmansetnosy: + ncoghlan
2020-12-29 03:15:45ethan.furmansetnosy: + rhettinger, stutzbach
messages: + msg383948
2020-12-29 03:08:11ethan.furmanlinkissue35815 superseder
2020-12-29 03:06:49ethan.furmancreate