classification
Title: MagicMock specialisation instance can no longer be passed to new MagicMock instance
Type: Stage:
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Elena.Oat, Frank Harrison, cjw296, lisroach, mariocj89, michael.foord, xtreak
Priority: normal Keywords:

Created on 2020-02-07 09:38 by Frank Harrison, last changed 2020-02-11 00:03 by Elena.Oat.

Messages (5)
msg361552 - (view) Author: Frank Harrison (Frank Harrison) Date: 2020-02-07 09:38
This is my first bug logged here, I've tried to follow the guideline and search for this issue; please let me know if I missed anything.

Summary:
unittest.mock.MagicMock has a regression starting in 3.8. The regression was only tested on latest non-prerelease versions of python 3.5, 3.6, 3.7, 3.8 and 3.9. Tested on OSX and Fedora 31.

Repro:
------
If you create an instance of a MagicMock specialisation with parameters to __init__(),  you can no longer pass that instance to the __init__() function of another MagicMock object e.g. a base-class is replaced with MagicMock. See the unittests bellow for more details, use-cases and fail situations.

What happens:
-------------
Here's a python3.9 example traceback. It may be worth noting that there is a difference in the tracebacks between 3.8 and 3.9.

Traceback (most recent call last):
  File "<...>", line <..>, in test_raw_magic_moc_passing_thru_single_pos
    mock_object = mock.MagicMock(mock_param)  # error is here, instantiating another object
  File "/usr/lib64/python3.9/unittest/mock.py", line 408, in __new__
    if spec_arg and _is_async_obj(spec_arg):
  File "/usr/lib64/python3.9/unittest/mock.py", line 2119, in __get__
    return self.create_mock()
  File "/usr/lib64/python3.9/unittest/mock.py", line 2112, in create_mock
    m = parent._get_child_mock(name=entry, _new_name=entry,
  File "/usr/lib64/python3.9/unittest/mock.py", line 1014, in _get_child_mock
    return klass(**kw)
TypeError: __init__() got an unexpected keyword argument 'name'


Code demonstrating the problem:
-------------------------------

import unittest

from unittest import mock


class TestMockMagicAssociativeHierarchies(unittest.TestCase):
    """ Mimicing real-world testing where we mock a base-class

    The intent here is to demonstrate some of the requirements of associative-
    hierarchies e.g. where a class may have its associative-parent set at
    run-time, rather that defining it via a class-hierarchy. Obviously this
    also needs to work with class-hierarchies, that is an associative-parent is
    likely to be a specialisation of some interface, usually one that is being
    mocked.

    For example tkinter and Qt have both a class-hierarchy and a layout-
    hierarchy; the latter is modifyable at runtime.

    Most of the tests here mimic a specialisation of an upstream object (say a
    tk.Frame class), instantiating that specialisation and then passing it to
    another object. The reason behind this is an observed regression in Python
    3.8.
    """
    def test_raw_magic_moc_passing_thru_no_params(self):
        """ REGRESSION: Python3.8 (inc Python3.9)

        Create a mocked specialisation passing it to another mock.

        One real-world use-case for this is simple cases where we simply want to
        define a new convenience type that forces a default configuration of
        the inherited type (calls super().__init__()).
        """
        class MockSubCallsParentInit(mock.MagicMock):
            def __init__(self):
                super().__init__()  # intentionally empty
        mock_param = MockSubCallsParentInit()
        mock_object = mock.MagicMock(mock_param)  # error is here, instantiating another object
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_raw_magic_moc_passing_thru_single_pos(self):
        """ REGRESSION: Python3.8 (inc Python3.9)

        Same as test_raw_magic_moc_no_init_params() but we want to specialise
        with positional arguments. """
        class MockSubCallsParentInitWithPositionalParam(mock.MagicMock):
            def __init__(self):
                super().__init__("specialise init calls")
        mock_param = MockSubCallsParentInitWithPositionalParam()
        mock_object = mock.MagicMock(mock_param)  # error is here, instantiating another object
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_raw_magic_moc_passing_thru_single_kwarg(self):
        """ REGRESSION: Python3.8 (inc Python3.9)

        Same as test_raw_magic_moc_passing_thru_single_pos() but we want to
        specialise with a key-word argument. """
        class MockSubCallsParentInitWithPositionalParam(mock.MagicMock):
            def __init__(self):
                super().__init__(__some_key_word__="some data")
        mock_param = MockSubCallsParentInitWithPositionalParam()
        mock_object = mock.MagicMock(mock_param)  # error is here, instantiating another object
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_mock_as_param_no_inheritance(self):
        """ PASSES 

        Mimic mocking out types, without type specialisation.
        for example in pseudo code 
            tk.Frame = mock.MagicMock; tk.Frame(t.Frame) """
        mock_param = mock.MagicMock()
        mock_object = mock.MagicMock(mock_param)
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_mock_as_param_no_init_override(self):
        """ PASSES 

        Leaves the __init__() function behaviour as default; should always
        work. Note that we do not specialise member functions.

        Although the intent here is similar to the one captured by
        test_raw_magic_moc_passing_thru_no_params(), this is a less likely
        usecase, although it does happen, but is here for completeness """
        class MockSub(mock.MagicMock): pass
        mock_param = MockSub()
        mock_object = mock.MagicMock(mock_param)
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_init_with_args_n_kwargs_passthru(self):
        """ PASSES

        Intended to be the same as test_mock_as_param_no_init_override as well
        as a base-test for ithe usecases where a user will define more complex
        behaviours such as key-word modification, member-variable definitions
        and so on. """
        class MockSubInitPassThruArgsNKwargs(mock.MagicMock):
            def __init__(self, *args, **kwargs):
                super().__init__(*args, **kwargs)  # intentionally redundant
        mock_param = MockSubInitPassThruArgsNKwargs()
        mock_object = mock.MagicMock(mock_param)
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_init_with_args_n_kwargs_modify_kwargs(self):
        """ PASSES

        Same as test_init_with_args_n_kwargs_passthru() but modifies the kwargs
        dict on the way through the __init__() function.
        """
        class MockSubModifyKwargs(mock.MagicMock):
            def __init__(self, *args, **kwargs):
                kwargs["__kw args added__"] = "test value"
                super().__init__(*args, **kwargs)
        mock_param = MockSubModifyKwargs()
        mock_object = mock.MagicMock(mock_param)
        self.assertIsInstance(mock_object, mock.MagicMock)

    def test_init_with_args_n_kwargs_modify_args(self):
        """ PASSES

        Same as test_init_with_args_n_kwargs_passthru() but modifies the args
        on their way through the __init__() function.
        """
        class MockSubModifyArgs(mock.MagicMock):
            def __init__(self, *args, **kwargs):
                super().__init__("test value", *args, **kwargs)
        mock_param = MockSubModifyArgs()
        mock_object = mock.MagicMock(mock_param)
        self.assertIsInstance(mock_object, mock.MagicMock)
msg361554 - (view) Author: Frank Harrison (Frank Harrison) Date: 2020-02-07 09:51
Minor correction: The regression was only tested on Python 3.9.0a2 (Fedora), Python 3.9a3 (OSX), CPython's master (build from source) and the latest non-prerelease versions of python 3.5, 3.6, 3.7, and 3.8. Tested on OSX and Fedora 31.
msg361657 - (view) Author: Mario Corchero (mariocj89) * (Python triager) Date: 2020-02-09 16:46
Having not looked deeply at it but with the reproducer, running a quick bisect, it points to this commit: https://github.com/python/cpython/commit/77b3b7701a34ecf6316469e05b79bb91de2addfa 
I'll try having a look later at the day if I can manage to free some time.
msg361723 - (view) Author: Elena Oat (Elena.Oat) * Date: 2020-02-10 21:29
I am looking at reproducing this and creating a short example of where this fails. Will look further into more details of why this doesn't work.
msg361763 - (view) Author: Elena Oat (Elena.Oat) * Date: 2020-02-11 00:03
Here's the example I ran, that indeed fails in Python 3.8 and Python 3.9 (with different errors) and works in Python 3.7. 

from unittest.mock import MagicMock


class CustomMock(MagicMock):
    def __init__(self):
        super().__init__(__something__='something')


mock = CustomMock()
MagicMock(mock)


In Python 3.8 the error is TypeError: __init__() got an unexpected keyword argument '_new_parent'. 

In Python 3.9 the error is TypeError: __init__() got an unexpected keyword argument 'name'.
History
Date User Action Args
2020-02-11 00:03:09Elena.Oatsetmessages: + msg361763
2020-02-10 21:29:20Elena.Oatsetnosy: + Elena.Oat
messages: + msg361723
2020-02-09 16:46:43mariocj89setmessages: + msg361657
2020-02-07 09:51:15Frank Harrisonsetmessages: + msg361554
2020-02-07 09:40:28xtreaksetnosy: + cjw296, michael.foord, lisroach, mariocj89, xtreak
2020-02-07 09:38:41Frank Harrisoncreate