classification
Title: Mocking a MagicMock with a function spec results in an AsyncMock
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: hroncok, jcline, lisroach, mariocj89, xtreak
Priority: normal Keywords: patch

Created on 2019-06-12 14:00 by jcline, last changed 2019-06-26 04:14 by lisroach.

Pull Requests
URL Status Linked Edit
PR 14117 open xtreak, 2019-06-15 16:34
Messages (7)
msg345358 - (view) Author: Jeremy Cline (jcline) * Date: 2019-06-12 14:00
This is related to the new AsyncMock[0] class in Python 3.8b1. A simple reproducer is:

from unittest import mock

mock_obj = mock.MagicMock()
mock_obj.mock_func = mock.MagicMock(spec=lambda x: x)

with mock.patch.object(mock_obj, "mock_func") as nested:
    print(type(nested))


Instead of a MagicMock (the behavior in Python 3.7) in Python 3.8b1 this results in an AsyncMock.

[0]https://github.com/python/cpython/pull/9296
msg345377 - (view) Author: Lisa Roach (lisroach) * (Python committer) Date: 2019-06-12 16:14
Following up from xtreak's proposal (https://github.com/python/cpython/pull/9296) I think checking if __code__ is actually a CodeType is a good idea. It's simple and doesn't change any other functionality in an unwanted way.
msg345399 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-06-12 18:16
I will wait for a couple of days for suggestions and will raise a PR to check for __code__ to be a CodeType. Thanks.
msg345704 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-06-15 16:53
I created PR to ensure the __code__ object is checked to be a CodeType and converted the report to a unittest. I also found a similar function _is_async_func [0] which also seems to perform similar check but is used only in create_autospec. creating an autospec function out of MagicMock with a function spec is not possible so though the change could be made it's not testable. Also changing _is_async_func to _is_async_obj in create_autospec shows no test case failure. Can this be removed to use only _is_async_obj? Is there a difference in their usage due to isawaitable check present in _is_async_obj that needs a test?

# create_autospec with MagicMock(spec=lambda x: x)

$ cpython git:(bpo37251) ./python.exe
Python 3.9.0a0 (heads/master:7a68f8c28b, Jun 15 2019, 21:00:05)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from unittest.mock import *
>>> create_autospec(MagicMock())
<MagicMock spec='MagicMock' id='4370353280'>
>>> create_autospec(MagicMock(spec=lambda x: x))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2547, in create_autospec
    mock = Klass(parent=_parent, _new_parent=_parent, _new_name=_new_name,
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 2066, in __init__
    super().__init__(*args, **kwargs)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 1996, in __init__
    _safe_super(AsyncMagicMixin, self).__init__(*args, **kw)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 1007, in __init__
    _safe_super(CallableMixin, self).__init__(
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 442, in __init__
    self._mock_add_spec(spec, spec_set, _spec_as_instance, _eat_self)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 503, in _mock_add_spec
    res = _get_signature_object(spec,
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 99, in _get_signature_object
    return func, inspect.signature(sig_func)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 3093, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 2842, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 2292, in _signature_from_callable
    return _signature_from_function(sigcls, obj,
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 2175, in _signature_from_function
    parameters.append(Parameter(name, annotation=annotation,
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 2495, in __init__
    raise TypeError(msg)
TypeError: name must be a str, not a MagicMock


[0] https://github.com/python/cpython/blob/3a1d50e7e573efb577714146bed5c03b9c95f466/Lib/unittest/mock.py#L55
msg346091 - (view) Author: Lisa Roach (lisroach) * (Python committer) Date: 2019-06-20 00:21
Thanks for the patch!

To answer your question, I do not think we can remove _is_async_func in favor of _is_async_obj, _is_async_obj will evaluate to True in cases where _is_async_func would not. 

For example:

>>> class NewCoroutine(Awaitable):
...     def __await__():
...         pass
...
>>> c = NewCoroutine()
>>> import inspect
>>> inspect.isawaitable(c)
True
>>> inspect.iscoroutinefunction(c)
False


BUT I think removing the `if getattr(obj, '__code__', None)` from `_is_async_obj` actually makes this work correctly. It is possible for a coroutine object to not have a __code__, but I don't think it is possible for a coroutine function to be missing a __code__. 

Before removing the __code__ check:

>>> from unittest.mock import _is_async_func, _is_async_obj
>>> import asyncio
>>> _is_async_obj(asyncio.sleep(1))
<stdin>:1: RuntimeWarning: coroutine 'sleep' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
False
>>> _is_async_func(asyncio.sleep(1))
False

_is_async_obj evaluates to False when it should be True

After removing it:

>>> from unittest.mock import _is_async_func, _is_async_obj
>>> import asyncio
>>> _is_async_obj(asyncio.sleep(1))
<stdin>:1: RuntimeWarning: coroutine 'sleep' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
True
>>> _is_async_func(asyncio.sleep(1))
False

It correctly evaluates to True

All tests pass as well. What do you think?
msg346150 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-06-20 18:11
> BUT I think removing the `if getattr(obj, '__code__', None)` from `_is_async_obj` actually makes this work correctly. It is possible for a coroutine object to not have a __code__, but I don't think it is possible for a coroutine function to be missing a __code__. 

Sorry, I am little confused here. If the code attribute check is removed then my test case in PR fails since obj.__code__ that is passed through iscoroutinefunction returns True. Maybe something along the lines of below that if there is a __code__ attribute then always check it's of CodeType. So that my test passes with MagicMock.__code__ detected.

If I understand the awaitable examples correctly, mocking the obj which is an Awaitable should be returning an AsyncMock. But obj doesn't contain __code__ and hence check for inspect.isawaitable is never done causing _is_async_obj(obj) to return False and subsequently it's patched with MagicMock.


from collections.abc import Awaitable
from unittest.mock import patch

class NewCoroutine(Awaitable):
    def __await__():
        pass

obj = NewCoroutine()

with patch(f"{__name__}.obj") as m:
    print(m)

$ ./python.exe ../backups/bpo37251_awaitable.py
<MagicMock name='obj' id='4552158896'>

On removing the __code__ attribute check my test case of MagicMock with __code__ passes through iscoroutinefunction. Perhaps an acceptable tradeoff would be to check for __code__ and if present to be a CodeType or else to resume normal check like below. This way an AsyncMock is returned. Also there is no test failure. I have less understanding on asyncio terminologies over coroutine and awaitables so feel free to correct me if I am wrong. I guess it would be also helpful to have good number of tests for different asyncio object cases so that this could also be documented.

$ ./python.exe ../backups/bpo37251_awaitable.py
<AsyncMock name='obj' id='4363294672'>

# _is_async_obj to check for __code__ to be CodeType only if present.

def _is_async_obj(obj):
    code = getattr(obj, '__code__', None)
    if code and not isinstance(code, CodeType):
        return False
    return asyncio.iscoroutinefunction(obj) or inspect.isawaitable(obj)

# Also verified asyncio.sleep() to return True for _is_async_obj with above definition

>>> from unittest.mock import _is_async_func, _is_async_obj
>>> import asyncio
>>> _is_async_obj(asyncio.sleep(1))
<stdin>:1: RuntimeWarning: coroutine 'sleep' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
True
msg346583 - (view) Author: Lisa Roach (lisroach) * (Python committer) Date: 2019-06-26 04:14
Yes, sorry I wasn't clear, I was thinking about the functions and testing without your PR. I think removing the __code__ object (or working around it) is the correct way to go, but just removing it wouldn't solve this particular problem.

"If I understand the awaitable examples correctly, mocking the obj which is an Awaitable should be returning an AsyncMock. But obj doesn't contain __code__ and hence check for inspect.isawaitable is never done causing _is_async_obj(obj) to return False and subsequently it's patched with MagicMock."

Exactly! This is why I think technically removing the __code__ check is correct. Probably removing the __code__ attribute for any AsyncMock that is mocking an async object and not an async function is best, but I don't know how I would do that. 

I may also be misunderstanding some asyncio concepts, that is just what I observed :)


What if instead of checking for the __code__ object at all we check if there it is a Mock object (excluding AsyncMock):
    

    def _is_async_obj(obj):
        sync_mocks = [MagicMock, Mock, PropertyMock, NonCallableMock, NonCallableMagicMock]
        if (any(isinstance(obj, sync_mock) for sync_mock in sync_mocks)
                and not isinstance(obj, AsyncMock)):
            return False
        return asyncio.iscoroutinefunction(obj) or inspect.isawaitable(obj)
History
Date User Action Args
2019-06-26 04:14:59lisroachsetmessages: + msg346583
2019-06-20 18:11:12xtreaksetmessages: + msg346150
2019-06-20 00:21:24lisroachsetmessages: + msg346091
2019-06-15 16:53:06xtreaksetmessages: + msg345704
versions: + Python 3.9
2019-06-15 16:34:36xtreaksetkeywords: + patch
stage: patch review
pull_requests: + pull_request13967
2019-06-12 18:16:29xtreaksetmessages: + msg345399
2019-06-12 16:14:41lisroachsetmessages: + msg345377
2019-06-12 15:05:33hroncoksetnosy: + hroncok
2019-06-12 14:11:44mariocj89setnosy: + mariocj89
2019-06-12 14:06:24xtreaksetnosy: + lisroach, xtreak
2019-06-12 14:00:18jclinecreate