Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unittest.mock.patch decorator doesn't work with async functions #81177

Closed
tirkarthi opened this issue May 21, 2019 · 6 comments
Closed

unittest.mock.patch decorator doesn't work with async functions #81177

tirkarthi opened this issue May 21, 2019 · 6 comments
Labels
3.8 only security fixes stdlib Python modules in the Lib dir topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@tirkarthi
Copy link
Member

BPO 36996
Nosy @cjw296, @voidspace, @asvetlov, @1st1, @lisroach, @miss-islington, @tirkarthi
PRs
  • bpo-36996: Handle async functions when mock.patch is used as a decorator #13562
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-05-28.07:10:01.250>
    created_at = <Date 2019-05-21.15:22:08.571>
    labels = ['3.8', 'type-bug', 'library', 'expert-asyncio']
    title = "unittest.mock.patch decorator doesn't work with async functions"
    updated_at = <Date 2019-05-28.07:10:01.250>
    user = 'https://github.com/tirkarthi'

    bugs.python.org fields:

    activity = <Date 2019-05-28.07:10:01.250>
    actor = 'asvetlov'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-05-28.07:10:01.250>
    closer = 'asvetlov'
    components = ['Library (Lib)', 'asyncio']
    creation = <Date 2019-05-21.15:22:08.571>
    creator = 'xtreak'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 36996
    keywords = ['patch']
    message_count = 6.0
    messages = ['343067', '343084', '343086', '343087', '343088', '343738']
    nosy_count = 7.0
    nosy_names = ['cjw296', 'michael.foord', 'asvetlov', 'yselivanov', 'lisroach', 'miss-islington', 'xtreak']
    pr_nums = ['13562']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue36996'
    versions = ['Python 3.8']

    @tirkarthi
    Copy link
    Member Author

    I came across this while using AsyncMock but it seems to apply to Mock/MagicMock too. When patch decorator is used to wrap an async function to mock sync or async functions it doesn't seem to work. Manually adding patcher or using patch as context manager works. Meanwhile sync_main which is not an async function works fine. Is this an expected behavior with @patch and async def? Does evaluating an async function with asyncio.run has any effect on this? Debugging it tells me the correct object is being replaced with AsyncMock inside patch.

    import asyncio
    from unittest.mock import patch, AsyncMock
    
    mock = AsyncMock()
    
    async def foo():
        pass
    
    def bar():
        pass

    @patch(f"{name}.foo", mock)
    @patch(f"{name}.bar", mock)
    async def main():
    print(f"Inside main {foo=}")
    patcher1 = patch(f"{name}.foo", mock)
    patcher2 = patch(f"{name}.bar", mock)
    print(f"Inside main before patch start {foo} {bar}")
    patcher1.start()
    patcher2.start()
    print(f"Inside main after patch start {foo} {bar}")
    await foo()
    with patch(f"{name}.foo", mock):
    with patch(f"{name}.bar", mock):
    print(f"Inside main with {foo} {bar}")
    await foo()

    @patch(f"{name}.foo", mock)
    @patch(f"{name}.bar", mock)
    def sync_main():
    print(f"Inside sync_main patch {foo} {bar}")
    with patch(f"{name}.foo", mock):
    with patch(f"{name}.bar", mock):
    print(f"Inside sync_main with {foo} {bar}")

    if __name__ == "__main__":
        asyncio.run(main())
        sync_main()

    ./python.exe foo.py
    Inside main foo=<function foo at 0x10f115af0>
    Inside main before patch start <function foo at 0x10f115af0> <function bar at 0x10f13f730>
    Inside main after patch start <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
    Inside main with <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
    Inside sync_main patch <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>
    Inside sync_main with <AsyncMock id='4541648720'> <AsyncMock id='4541648720'>

    @tirkarthi tirkarthi added 3.8 only security fixes stdlib Python modules in the Lib dir topic-asyncio type-bug An unexpected behavior, bug, or error labels May 21, 2019
    @asvetlov
    Copy link
    Contributor

    Thank you very much for raising the question.

    @patch(...) creates _patch class instance.
    For decoration _patch.__call__ is used.

        def __call__(self, func):
            if isinstance(func, type):
                return self.decorate_class(func)
            return self.decorate_callable(func)

    The code can be modified to

        def __call__(self, func):
            if isinstance(func, type):
                return self.decorate_class(func)
            if inspect.iscoroutinefunction(func):
                return self.decorate_async_func(func)
            return self.decorate_callable(func)

    decorate_async_func can do all the same as decorate_callable with the only difference: internal
    @wraps(func)
    def patched(*args, **keywargs):
    should be replaced with
    @wraps(func)
    async def patched(*args, **keywargs):
    and
    return func(*args, **keywargs)
    replaced with
    return await func(*args, **keywargs)

    Pretty straightforward.

    I did not check the code though.
    I'm pretty busy up to 3.8 feature freeze to make the PR for proposed change but I love to review the existing patch.

    Do want somebody to be a champion?

    @tirkarthi
    Copy link
    Member Author

    Thanks @asvetlov for the explanation. I tried the patch and it works fine for my example with no test failures for mock. I will try to make a PR. Only func(*args, **keywargs) needs to be changed and await requires an async function which I seem to need duplicating the function. I will try if I can refactor that. I will post a PR where this can be discussed.

    @lisroach
    Copy link
    Contributor

    I quickly threw in Andrew's code to check it and looks like it does fix the problem. I'd be happy to add it in but I'll give @XTreak first dibs.

    Thanks for finding this :)

    @lisroach
    Copy link
    Contributor

    Oops, didn't see your post. Thanks!

    @miss-islington
    Copy link
    Contributor

    New changeset 436c2b0 by Miss Islington (bot) (Xtreak) in branch 'master':
    bpo-36996: Handle async functions when mock.patch is used as a decorator (GH-13562)
    436c2b0

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes stdlib Python modules in the Lib dir topic-asyncio type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants