classification
Title: mock calls don't propagate to parent (autospec)
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: Claudiu.Popa, and, cjw296, iforapsy, kushal.das, mariocj89, michael.foord, xtreak
Priority: normal Keywords: patch

Created on 2014-05-12 12:56 by and, last changed 2019-07-11 20:20 by iforapsy.

Pull Requests
URL Status Linked Edit
PR 11273 merged xtreak, 2018-12-21 06:45
PR 12039 merged miss-islington, 2019-03-03 17:14
PR 14688 open xtreak, 2019-07-10 18:32
Messages (7)
msg218321 - (view) Author: Dmitry Andreychuk (and) Date: 2014-05-12 12:56
Calls to autospecced mock functions are not recorded to mock_calls list of parent mock. This only happens if autospec is used and the original object is a function.

Example:

import unittest.mock as mock

def foo():
    pass

parent = mock.Mock()
parent.child = mock.create_autospec(foo)
parent.child()
print(parent.mock_calls)


Output:
[]

Expected output:
[call.child()]

It works fine if foo function is substituted with a class.

Initially I came across this problem with patch() and attach_mock() but I simplified it for the demonstration.
msg218362 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-05-12 20:48
Mock objects detect when another mock is added as a "child", but they don't currently detect that a function created by autospec has been added. It should be a fairly easy fix.
msg332252 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2018-12-20 19:45
I think this should be handled in _check_and_set_parent where if value's type is FunctionType then value.mock should be used against which parent and name should be set since create_autospec returns function with mock attached to 'mock' attribute and all helper methods attached. There could be a case where a function is directly attached to mock where value will be of FunctionType but value.mock will not be set since it's not created with create_autospec and the AttributeError has to silenced. I think the below patch will handle both scenarios. This doesn't cause any test failure and hence it will be good to convert the original report as a unit test. Feedback welcome on the approach. I will raise a PR with tests and I am updating the relevant versions where this fix can be applied.

Sample program

from unittest import mock

def foo(a, b):
    pass

parent = mock.Mock()
a = mock.create_autospec(foo)
parent.child = a # 'a' is FunctionType and has a.mock attribute set (create_autospec -> _set_signature -> _setup_func)

parent.child_func = foo # 'foo' is FunctionType with no mock attribute set that could cause AttributeError

parent.child(1, 2) # Recorded
parent.child_func(2, 3) # Not recorded since it's actual call to child_func and has no parent set due to AttributeError
print(parent.method_calls) # [call.child(1, 2)]

Patch : 

diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py
index 38189c9aec..143263722d 100644
--- a/Lib/unittest/mock.py
+++ b/Lib/unittest/mock.py
@@ -321,6 +321,12 @@ class _CallList(list):


 def _check_and_set_parent(parent, value, name, new_name):
+    if isinstance(value, FunctionTypes):
+        try:
+            value = value.mock
+        except AttributeError:
+            pass
+
     if not _is_instance_mock(value):
         return False
     if ((value._mock_name or value._mock_new_name) or
msg347592 - (view) Author: Jack Wong (iforapsy) Date: 2019-07-10 01:50
Can we reopen this bug? Karthikeyan's PR works for Dmitry's toy example, but it does not work in the usual case where patch() and attach_mock() are used. I encountered this bug on Python 3.7.3, which includes the PR.

Non-toy example:
    import unittest.mock as mock

    def foo():
        pass

    parent = mock.Mock()

    with mock.patch('__main__.foo', autospec=True) as mock_foo:
        parent.attach_mock(mock_foo, 'child')
        parent.child()
        print(parent.mock_calls)

Actual output:
[]

Expected output:
[call.child()]

The reason why Karthikeyan's PR works on the toy example is that that mock's name is not set. In the usual case, the function mock's name will be set so this "if" block in _check_and_set_parent will return immediately.
    if ((value._mock_name or value._mock_new_name) or
        (value._mock_parent is not None) or
        (value._mock_new_parent is not None)):
        return False

I think a possible fix is to move the inner mock extraction out to the attach_mock function as that function contains code to clear the mock's parent and name attributes. Downside is that that would make it fail on Dmitry's toy example.
msg347593 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-07-10 02:22
Thanks Jack for the report. I am reopening this issue. I will use your example as a unit test. I will try to look into it. If you have more cases or examples related to the issue feel free to add them.
msg347631 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-07-10 18:38
> I think a possible fix is to move the inner mock extraction out to the attach_mock function as that function contains code to clear the mock's parent and name attributes. Downside is that that would make it fail on Dmitry's toy example.

Moving the inner mock check would cause the original report to fail. But the same logic can be copied to attach_mock so that name and parent for inner mock set in "mock" attribute is cleared. So _check_and_set_parent code path is hit the mock attribute without name and parent would behave as expected. Jack, would appreciate your review of the PR. I have added your report as a unittest along with directly using create_autospec with attach_mock that behaves like patch and autospec=True. I have also asserted that name is reset to child as per the docs at https://docs.python.org/3/library/unittest.mock.html#attaching-mocks-as-attributes

Thanks
msg347705 - (view) Author: Jack Wong (iforapsy) Date: 2019-07-11 20:20
Thanks, Karthikeyan! I applied your change to Lib/unittest/mock.py, and it fixed the bug for me. I confirmed that calls to the child mock now appear in the parent's mock_calls in my test suite. Your PR looks good to me.
History
Date User Action Args
2019-07-11 20:20:01iforapsysetmessages: + msg347705
2019-07-10 18:38:18xtreaksetmessages: + msg347631
2019-07-10 18:32:26xtreaksetstage: patch review
pull_requests: + pull_request14493
2019-07-10 02:22:05xtreaksetstatus: closed -> open
versions: + Python 3.9
messages: + msg347593

resolution: fixed ->
stage: resolved -> (no value)
2019-07-10 01:50:13iforapsysetnosy: + iforapsy
messages: + msg347592
2019-03-03 17:31:37xtreaksetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-03-03 17:14:28miss-islingtonsetpull_requests: + pull_request12147
2018-12-21 06:45:10xtreaksetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request10508
2018-12-20 19:45:55xtreaksetnosy: + cjw296, mariocj89, xtreak

messages: + msg332252
versions: + Python 3.7, Python 3.8, - Python 3.3, Python 3.4, Python 3.5
2015-03-09 14:44:35Claudiu.Popasetnosy: + Claudiu.Popa
2014-05-12 20:49:31michael.foordsetnosy: + kushal.das
stage: needs patch

versions: + Python 3.5
2014-05-12 20:48:16michael.foordsetassignee: michael.foord
messages: + msg218362
2014-05-12 20:46:38ned.deilysetnosy: + michael.foord
2014-05-12 12:56:34andcreate