This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author vstinner
Recipients eric.smith, petr.viktorin, rhettinger, serhiy.storchaka, vstinner
Date 2022-03-28.13:51:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1648475497.69.0.517848943478.issue47143@roundup.psfhosted.org>
In-reply-to
Content
Class decorarators of attrs and stdlib dataclasses modules have to copy a class to *add* slots:

* old fixed attrs issue: https://github.com/python-attrs/attrs/issues/102
* attrs issue with Python 3.11: https://github.com/python-attrs/attrs/issues/907
* dataclasses issues with slots=True: https://bugs.python.org/issue46404


In the common case, copying a class is trivial:

    cls2 = type(cls)(cls.__name__, cls.__bases__, cls.__dict__)

Full dummy example just to change a class name without touching the original class (create a copy with a different name):
---
class MyClass:
    def hello(self):
        print("Hello", self.__class__)

def copy_class(cls, new_name):
    cls_dict = cls.__dict__.copy()
    # hack the dict to modify the class copy
    return type(cls)(new_name, cls.__bases__, cls_dict)

MyClass2 = copy_class(MyClass, "MyClass2")
MyClass2().hello()
---

Output:
---
Hello <class '__main__.MyClass2'>
---


The problem is when a class uses a closure ("__class__" here):
---
class MyClass:
    def who_am_i(self):
        cls = __class__
        print(cls)
        if cls is not self.__class__:
            raise Exception(f"closure lies: __class__={cls} {self.__class__=}")

def copy_class(cls, new_name):
    cls_dict = cls.__dict__.copy()
    # hack the dict to modify the class copy
    return type(cls)(new_name, cls.__bases__, cls_dict)

MyClass().who_am_i()
MyClass2 = copy_class(MyClass, "MyClass2")
MyClass2().who_am_i()
---

Output:
---
<class '__main__.MyClass'>
<class '__main__.MyClass'>
Traceback (most recent call last):
  ...
Exception: closure lies: __class__=<class '__main__.MyClass'> self.__class__=<class '__main__.MyClass2'>
---


The attrs project uses the following complicated code to workaround this issue (to "update closures"):
---

        # The following is a fix for
        # <https://github.com/python-attrs/attrs/issues/102>.  On Python 3,
        # if a method mentions `__class__` or uses the no-arg super(), the
        # compiler will bake a reference to the class in the method itself
        # as `method.__closure__`.  Since we replace the class with a
        # clone, we rewrite these references so it keeps working.
        for item in cls.__dict__.values():
            if isinstance(item, (classmethod, staticmethod)):
                # Class- and staticmethods hide their functions inside.
                # These might need to be rewritten as well.
                closure_cells = getattr(item.__func__, "__closure__", None)
            elif isinstance(item, property):
                # Workaround for property `super()` shortcut (PY3-only).
                # There is no universal way for other descriptors.
                closure_cells = getattr(item.fget, "__closure__", None)
            else:
                closure_cells = getattr(item, "__closure__", None)

            if not closure_cells:  # Catch None or the empty list.
                continue
            for cell in closure_cells:
                try:
                    match = cell.cell_contents is self._cls
                except ValueError:  # ValueError: Cell is empty
                    pass
                else:
                    if match:
                        set_closure_cell(cell, cls)
---
source: https://github.com/python-attrs/attrs/blob/5c040f30e3e4b3c9c0f27c8ac6ff13d604c1818c/src/attr/_make.py#L886

The implementation of the set_closure_cell() function is really complicate since cells were mutable before Python 3.10:
https://github.com/python-attrs/attrs/blob/5c040f30e3e4b3c9c0f27c8ac6ff13d604c1818c/src/attr/_compat.py#L203-L305


I propose to add a new functools.copy_class() function which copy a class and update the closures: end of the _create_slots_class() function:
---
cls = type(self._cls)(...)
for item in cls.__dict__.values():
    ... # update closures
return cls
---


The alternative is not to add a function to copy a class, just only to "update closures", but IMO such API would be more error prone.


I would like to implement this function, but first I would like to dicuss if it makes sense to add such function and check if it's the right abstraction.
History
Date User Action Args
2022-03-28 13:51:37vstinnersetrecipients: + vstinner, rhettinger, eric.smith, petr.viktorin, serhiy.storchaka
2022-03-28 13:51:37vstinnersetmessageid: <1648475497.69.0.517848943478.issue47143@roundup.psfhosted.org>
2022-03-28 13:51:37vstinnerlinkissue47143 messages
2022-03-28 13:51:37vstinnercreate