classification
Title: unittest.mock.patch decorator doesn't work with async functions
Type: behavior Stage: resolved
Components: asyncio, Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, cjw296, lisroach, michael.foord, miss-islington, xtreak, yselivanov
Priority: normal Keywords: patch

Created on 2019-05-21 15:22 by xtreak, last changed 2019-05-28 07:10 by asvetlov. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 13562 merged xtreak, 2019-05-25 06:31
Messages (6)
msg343067 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-05-21 15:22
I came across this while using AsyncMock but it seems to apply to Mock/MagicMock too. When patch decorator is used to wrap an async function to mock sync or async functions it doesn't seem to work. Manually adding patcher or using patch as context manager works. Meanwhile sync_main which is not an async function works fine. Is this an expected behavior with @patch and async def? Does evaluating an async function with asyncio.run has any effect on this? Debugging it tells me the correct object is being replaced with AsyncMock inside patch.

import asyncio
from unittest.mock import patch, AsyncMock

mock = AsyncMock()

async def foo():
    pass

def bar():
    pass

@patch(f"{__name__}.foo", mock)
@patch(f"{__name__}.bar", mock)
async def main():
    print(f"Inside main {foo=}")
    patcher1 = patch(f"{__name__}.foo", mock)
    patcher2 = patch(f"{__name__}.bar", mock)
    print(f"Inside main before patch start {foo} {bar}")
    patcher1.start()
    patcher2.start()
    print(f"Inside main after patch start {foo} {bar}")
    await foo()
    with patch(f"{__name__}.foo", mock):
        with patch(f"{__name__}.bar", mock):
            print(f"Inside main with {foo} {bar}")
            await foo()

@patch(f"{__name__}.foo", mock)
@patch(f"{__name__}.bar", mock)
def sync_main():
    print(f"Inside sync_main patch {foo} {bar}")
    with patch(f"{__name__}.foo", mock):
        with patch(f"{__name__}.bar", mock):
            print(f"Inside sync_main with {foo} {bar}")


if __name__ == "__main__":
    asyncio.run(main())
    sync_main()


./python.exe foo.py
Inside main foo=<function foo at 0x10f115af0>
Inside main before patch start <function foo at 0x10f115af0> <function bar at 0x10f13f730>
Inside main after patch start <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
Inside main with <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
Inside sync_main patch <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
Inside sync_main with <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
msg343084 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2019-05-21 19:06
Thank you very much for raising the question.

@patch(...) creates _patch class instance.
For decoration _patch.__call__ is used.

    def __call__(self, func):
        if isinstance(func, type):
            return self.decorate_class(func)
        return self.decorate_callable(func)

The code can be modified to

    def __call__(self, func):
        if isinstance(func, type):
            return self.decorate_class(func)
        if inspect.iscoroutinefunction(func):
            return self.decorate_async_func(func)
        return self.decorate_callable(func)

decorate_async_func can do all the same as decorate_callable with the only difference: internal
        @wraps(func)
        def patched(*args, **keywargs):
should be replaced with
        @wraps(func)
        async def patched(*args, **keywargs):
and
                return func(*args, **keywargs)
replaced with
                return await func(*args, **keywargs)

Pretty straightforward.

I did not check the code though.
I'm pretty busy up to 3.8 feature freeze to make the PR for proposed change but I love to review the existing patch.

Do want somebody to be a champion?
msg343086 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2019-05-21 19:29
Thanks @asvetlov for the explanation. I tried the patch and it works fine for my example with no test failures for mock. I will try to make a PR. Only func(*args, **keywargs) needs to be changed and await requires an async function which I seem to need duplicating the function. I will try if I can refactor that. I will post a PR where this can be discussed.
msg343087 - (view) Author: Lisa Roach (lisroach) * (Python committer) Date: 2019-05-21 19:30
I quickly threw in Andrew's code to check it and looks like it does fix the problem. I'd be happy to add it in but I'll give @xtreak first dibs.

Thanks for finding this :)
msg343088 - (view) Author: Lisa Roach (lisroach) * (Python committer) Date: 2019-05-21 19:30
Oops, didn't see your post. Thanks!
msg343738 - (view) Author: miss-islington (miss-islington) Date: 2019-05-28 07:07
New changeset 436c2b0d67da68465e709a96daac7340af3a5238 by Miss Islington (bot) (Xtreak) in branch 'master':
bpo-36996: Handle async functions when mock.patch is used as a decorator (GH-13562)
https://github.com/python/cpython/commit/436c2b0d67da68465e709a96daac7340af3a5238
History
Date User Action Args
2019-05-28 07:10:01asvetlovsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-05-28 07:07:55miss-islingtonsetnosy: + miss-islington
messages: + msg343738
2019-05-25 06:31:24xtreaksetkeywords: + patch
stage: patch review
pull_requests: + pull_request13473
2019-05-21 19:30:49lisroachsetmessages: + msg343088
2019-05-21 19:30:15lisroachsetmessages: + msg343087
2019-05-21 19:29:46xtreaksetmessages: + msg343086
2019-05-21 19:06:31asvetlovsetmessages: + msg343084
2019-05-21 15:22:08xtreakcreate