This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: The variable __module__ in the class body getting an undesirable value from __prepare__ of the metaclass
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Takuo Matsuoka, ethan.furman, steven.daprano
Priority: normal Keywords:

Created on 2022-03-27 11:17 by Takuo Matsuoka, last changed 2022-04-11 14:59 by admin.

Messages (10)
msg416118 - (view) Author: Takuo Matsuoka (Takuo Matsuoka) Date: 2022-03-27 11:17
In the creation of a class, it looks as if the value of the variable
__name__ gets assigned automatically to the variable __module__ in the
class body.  However, the correct name space where the value of
__name__ should be looked up is NOT the mapping object returned by the
method __prepare__
of the metaclass.  In the class body, the programmer may use the
variable __name__ for some other purposes, and might not notice
__module__ was messed up.  Here's a code which produces a problem.

```
class C(type):
    @classmethod
    def __prepare__(cls, /, *args, **kwargs):
        return dict(__name__ = "whatever")

class O(metaclass=C):
    print(__module__) # "whatever" printed
```

Consequently,

>>> O.__module__
'whatever'

The issue is different from but seems related to

https://bugs.python.org/issue28869

I haven't figured out the exact relation.

Thanks.
msg416136 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2022-03-27 21:35
> the programmer may use the variable __name__ for some other purposes

Dunder names like `__name__` are reserved for the use of the interpreter. If the programmer uses them for "other purposes", the programmer is responsible for any failures.

You wouldn't write:

    class MyList(list):
        __len__ = "Hello world!"

and then be surprised that MyList is broken. You shouldn't be surprised if setting `__name__` to an invalid value breaks things either.
msg416143 - (view) Author: Takuo Matsuoka (Takuo Matsuoka) Date: 2022-03-28 04:01
Thank you for your response.

I think __name__ here is very different from __len__ .

(1) Even if you set __name__ to what should be the right value, e.g.,
when my class O will be a subclass of say 'type', and __name__ is an
appropriate thing for your purposes to override the attribute
'__name__' of 'type' objects (held as the entry '__name__' of
type.__dict__), the same value is going to be O.__dict__["__module__"]
as long as you let __prepare__ of the mataclass C provide it.


(2) Even if, in other cases, you do

del __name__

at the end of the body of the class O, the problem remains unless you
know O.__dict__["__module__"] will be changed anyway.


The behaviour is not documented as far as I could see.  I think such a
behaviour is problematic if it can't be expected from what's
documented.  If the value of __name__ can be read from the scope
outside, then that will erase this unexpected behaviour and I think it
would be a much safer thing to do.

Thanks.
msg416743 - (view) Author: Takuo Matsuoka (Takuo Matsuoka) Date: 2022-04-05 03:39
I'm going to close this one since I failed to specify the issue
clearly enough.  See

https://bugs.python.org/issue47224

for a more specific issue.
msg416745 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2022-04-05 03:51
issue47223 and issue47224 closed, reopening this one.
msg416746 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2022-04-05 03:52
Takuo, please give us an example from real code so we can see the problem.
msg416769 - (view) Author: Takuo Matsuoka (Takuo Matsuoka) Date: 2022-04-05 11:04
Thank you Ethan for reopening this issue and closing the other one.

Here is a description of a more specific issue, containing a more
reasonable example.  I've changed the title of the issue to a more
appropriate one accordingly.

Context
-------
Some classes have the variable __name__ in their namespace __dict__ ,
and one may wish to create more such classes with varied values of
__name__ .  Some of those could be created with a metaclass whose
__prepare__ returns a mapping having key "__name__", for which the
value is created depending on the arguments of __prepare__ and can be
updated or deleted in the body of the class to be created.  (See C
below for a very silly example of such a metaclass.)


Problem
-------
The value of __name__ given by __prepare__ becomes not just that in
the class body, but automatically also the value of __module__
there.  As far as I could see, this is not documented, and the
programmer might not notice __module__ was messed up.  I think this
behaviour is unexpected and problematic at least unless a warning is
given on it in the document.

Also, the problem means we can't safely enjoy the ability of
__prepare__ of a metaclass to give a candidate for the value of
__name__ in __dict__ of the class without the trouble of fixing the
variable __module__ later at the top of the class body for every
instance of the metaclass (very annoying) somehow (or in __new__ or
__init__ of the metaclass if __module__ is not to be read in the class
body).


Example
-------
Here's a code which produces a problem.

```
# In this example, the metaclass C is intended to be a class of
# subclasses of:
B = type

class C(type(B)):
    @classmethod
    def __prepare__(cls, /, *args, **kwargs):
        return dict(__name__ = cls._name(*args, **kwargs))
    @classmethod
    def _name(cls, /, *args, **kwargs):
        # The actual value of __name__ doesn't matter much to the
        # issue, so I make this function always return the same silly
        # thing in this example.
        return type.__dict__["__name__"]

class O(B, metaclass=C):
    print(__module__ == __name__) # True
    # Could update or delete __name__ here.
```

Consequently,

>>> O.__module__
<attribute '__name__' of 'type' objects>


Thanks.

P.S.
The argument mentioning "the scope outside" in my earlier post here
didn't make sense without specifying which scope.  I still hope the
problem can be fixed.
msg416789 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2022-04-05 17:05
You are using the same generic example -- it does show the issue you are concerned with, but offers no rational for why we should "fix" it -- in other words, we cannot tell if it's actually broken.

Please provide an example of code you are actually using -- a trimmed down metaclass is fine, but it should show the kinds of names you are changing to, why you want to change the names, etc.

Maybe this is a just a documentation issue, or maybe it's an overlooked case that should be supported in Python -- we cannot tell based solely on your generic example.
msg416914 - (view) Author: Takuo Matsuoka (Takuo Matsuoka) Date: 2022-04-07 05:12
Thank you Ethan for your comments.

Sure, I was not familiar with how you measure the magnitude of the
consequences.  The code in my own work was of the kind of the generic
example I gave, but I have abandoned the approach, and don't seem able
to find it any more.  I think the approach turned out not ideal for
the specific aim I had, which I'm sorry I can't recall now.

I might just say there may be circumstances where a not so thoughtful
programmer might get inclined to implementing a metaclass C in some
manner like:

```
class C(type(B)):
    # Skip __init__ .  It's just to help checking type later.
    def __init__(self, /, *args, **kwargs):
        super().__init__(*args, **kwargs)
        dict_ = self.__dict__
        try:
            name = dict_["__name__"]
        except KeyError:
            pass
        else:
            name._owner = self
    @classmethod
    def __prepare__(cls, /, *args, **kwargs):
        return dict(__name__ = cls._name(*args, **kwargs))
    @classmethod
    class _name:
        def __get__(self, instance, owner=None):
            if instance is None:
                if issubclass(owner, self_owner := self._owner):
                    return self
                else:
                    raise TypeError(f'{owner} is not a subclass of'
                                    f' {self_owner.__qualname__}')
            name = instance._super().__name__ # See the class O
            # below. 
            #
            #
            # Any procedure here, depending on what you'd like to do
            # with the instance of your class...
            #
            return name
        def __init__(self, cls, /, *args, **kwargs): ......
        def __set__(self, instance, value): ......
        def __delete__(self, instance): ......
```

where she creates instances of the metaclass C by inheriting from:

```
class O(B, metaclass=C):
    def _super(self):
        return super()
    def __init_subclass__(cls, /, *args, **kwargs):
        return super().__init_subclass__(*args)
```

Another thing I can say is code that does something like that is not
what I write often or even had written before, I guess.  Still, I
reported the issue thinking some people (possibly including myself)
may come around the kind of code some time in the future again.

If the behaviour is not going to be changed, then I think the
documentation should at least be made sure to warn about it.  I don't
think the behaviour can be expected without documentation.

Thanks.
msg416924 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2022-04-07 11:50
It would be nice if the class creation process was documented a bit 
better. Here is my experiment to see what is going on:

```
# the global `__name__` is normally the module's name.
__name__ = "something"

class Meta_default_prepare(type):
    def __new__(meta, name, bases, ns):
        print("ns for", name, "\n   ", ns)
        return super().__new__(meta, name, bases, ns)

class Meta_custom_prepare(Meta_default_prepare):
    def __prepare__(meta, *args):
        return {'__name__': 'another_name'}

class Spam(metaclass=Meta_default_prepare):
    pass

class Eggs(metaclass=Meta_custom_prepare):
    pass

print("Spam module and name:", Spam.__module__, Spam.__name__)
print("Eggs module and name:", Eggs.__module__, Eggs.__name__)

```

And the output in Python 3.10 is:

```
ns for Spam 
    {'__module__': 'something', '__qualname__': 'Spam'}
ns for Eggs 
    {'__name__': 'another_name', '__module__': 'another_name', '__qualname__': 'Eggs'}
Spam module and name: something Spam
Eggs module and name: another_name Eggs
```

My take on this is that if the key __name__ is not present, the value of 
the class __module__ is taken from the global variable. So far so good.

But if '__name__' is a key in the mapping returned by __prepare__, it 
gets left in the class dict, and gets used to set the class __module__ 
as well.

But in neither case does it get used to set the class __name__.

I cannot decide whether or not this makes sense to me.
History
Date User Action Args
2022-04-11 14:59:57adminsetgithub: 91292
2022-04-07 11:50:41steven.dapranosetmessages: + msg416924
2022-04-07 05:12:24Takuo Matsuokasetmessages: + msg416914
2022-04-05 17:05:53ethan.furmansetmessages: + msg416789
2022-04-05 11:04:35Takuo Matsuokasetmessages: + msg416769
title: Wrong value assigned automatically to the variable __module__ in the class body. -> The variable __module__ in the class body getting an undesirable value from __prepare__ of the metaclass
2022-04-05 03:52:38ethan.furmansetmessages: + msg416746
2022-04-05 03:51:41ethan.furmansetstatus: closed -> open

nosy: + ethan.furman
messages: + msg416745

resolution: duplicate ->
stage: resolved ->
2022-04-05 03:39:23Takuo Matsuokasetstatus: open -> closed
resolution: duplicate
messages: + msg416743

stage: resolved
2022-03-28 04:01:17Takuo Matsuokasetmessages: + msg416143
2022-03-27 21:35:28steven.dapranosetnosy: + steven.daprano
messages: + msg416136
2022-03-27 11:17:56Takuo Matsuokacreate