classification
Title: User-created types with wrong __new__ can be instantiated
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Vadim Pushtaev, eric.snow, ncoghlan, ppperry, serhiy.storchaka
Priority: normal Keywords:

Created on 2018-08-08 19:07 by ppperry, last changed 2018-08-09 17:52 by ppperry.

Messages (4)
msg323297 - (view) Author: (ppperry) Date: 2018-08-08 19:07
If you have a class that defines __new__ to the __new__ of another builtin type that it isn't a subclass of:
>>> class X:
...    __new__ = tuple.__new__
Instantiating this class should produce an error because `tuple.__new__` can't handle non-tuples, but instead it succeeds:
>>> X()
<__main__.X object at 0x00000000032C3F98>

(related: issue34284)
msg323299 - (view) Author: (ppperry) Date: 2018-08-08 19:47
Whoops, realized this is a duplicate of issue5322.
msg323336 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2018-08-09 15:22
I'm not sure that this is a duplicate of #5322.  That one's about a warning when arguments are passed to object.__new__().  Yours is about using an incompatible __new__ when creating a new class in Python.  I agree that behavior you're seeing is unexpected and should probably be fixed.

Let's look at the situation a little more closely.  Here's what I understood you reported:

    >>> class X:
    ...   __new__ = tuple.__new__
    ...
    >>> x1 = X()  # This does not fail!

However, that is a little misleading because you might think it is calling tuple.__new__().  It isn't.  Apparently type.__call__() is invoking object.__new__() instead of X.__new__():

    >>> x2 = X([1, 2])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: object() takes no parameters
    >>> object.__new__(X, [1, 2])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: object() takes no parameters

If I had a little more time I'd look into the relevant code to see what's going on.  In the meantime, let's see if we can find the edges of this problem.

We can verify that X.__new__ is still set to tuple.__new__:

    >>> X.__new__ is object.__new__
    False
    >>> X.__new__ is tuple.__new__
    True
    >>> X.__new__(X)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: tuple.__new__(X): X is not a subtype of tuple
    >>> X.__new__(X, [1, 2])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: tuple.__new__(X): X is not a subtype of tuple
    >>> X.__new__(tuple)
    ()
    >>> X.__new__(tuple, [1, 2])
    (1, 2)

If we explicitly defer to tuple.__new__() then we get an error that matches our expectations better:

    >>> class Y:
    ...   def __new__(cls, *args, **kwargs):
    ...     return tuple.__new__(cls, *args, **kwargs)
    ...
    >>> y = Y()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 3, in __new__
    TypeError: tuple.__new__(Y): Y is not a subtype of tuple

This reinforces the conclusion that tuple.__call__() is doing something funny here.  We can take that conclusion further by seeing that the unexpected behavior is not specific to using tuple.__new__:

    >>> class Z:
    ...   __new__ = int.__new__
    ...
    >>> z1 = Z()
    >>> z2 = Z(42)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: object() takes no parameters
    >>> Z.__new__(Z)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: int.__new__(Z): Z is not a subtype of int
msg323347 - (view) Author: (ppperry) Date: 2018-08-09 17:52
issue5322, despite its confusing title, mentions this exact bug in one of the comments below. 

It looks like there is one bug in the logic for assigning `__new__`, which causes `__new__` and `tp_new` to point to different things, confusing the error-handling logic in `object.__new__` as well as causing creation of a type by calling it to work in cases where direct calls to __new__ fail.

There's no one bug report about that, however issue5322, issue25731 and some cases of issue34284 are all symptoms of the same root issue.
History
Date User Action Args
2018-08-09 17:52:28ppperrysetmessages: + msg323347
2018-08-09 15:22:13eric.snowsetnosy: + eric.snow
messages: + msg323336
2018-08-08 19:47:19ppperrysetmessages: + msg323299
2018-08-08 19:07:33ppperrycreate