classification
Title: AsyncMock is unable to correctly patch static or class methods
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, cjw296, czardoz, lisroach, mkokotovich, xtreak, yselivanov
Priority: normal Keywords: patch

Created on 2019-12-18 00:36 by czardoz, last changed 2020-01-26 15:30 by cjw296. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 17717 closed czardoz, 2019-12-27 20:31
PR 18116 merged python-dev, 2020-01-22 14:36
PR 18190 merged mkokotovich, 2020-01-26 02:33
Messages (7)
msg358601 - (view) Author: Aniket Panse (czardoz) * Date: 2019-12-18 00:36
Currently, patch is unable to correctly patch coroutinefunctions decorated with `@staticmethod` or `@classmethod`.

Example:

```
[*] aniketpanse [~/git/cpython] -> ./python                                                                                                                                                                                                         ±[master]
Python 3.9.0a1+ (heads/master:50d4f12958, Dec 17 2019, 16:31:30) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Helper:
...     @classmethod
...     async def async_class_method(cls):
...         pass
... 
>>> from unittest.mock import patch
>>> patch("Helper.async_class_method")
<unittest.mock._patch object at 0x7fc28ddbbf40>
```

This should ideally return an `AsyncMock()`.
msg358609 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-18 03:21
Using patch as a function just returns a patch object. You need to call start method on the patch object to return a Mock.
msg358612 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-18 05:52
There seems to be a difference in obtaining the attribute out of the __dict__ and getattr. patch uses __dict__ [0] to access the attribute to be patched and falls back to getattr. There is a difference in detecting normal attribute access to be a coroutine function versus the one obtained from __dict__ . We can perhaps make _is_async_obj [1] to see if the passed obj is a classmethod/staticmethod and to use __func__ to detect a coroutine function.

As a workaround for now you can pass AsyncMock explicitly to patch to return an AsyncMock. There was an open issue (issue36092) about patch and staticmethods/classmethods but not sure this is related to this.

Retagging it to remove asyncio. Thanks for the report!

# bpo39082.py

from unittest.mock import patch
import inspect

class Helper:

    @classmethod
    async def async_class_method(cls):
        pass

    @staticmethod
    async def async_static_method(*args, **kwargs):
        pass


print("Patching async static method")
async_patcher = patch("__main__.Helper.async_class_method")
print(f"{Helper.async_class_method = }")
print(f"{Helper.__dict__['async_class_method'] = }")
print(f"{inspect.iscoroutinefunction(Helper.async_class_method) = }")
print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = }")
print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = }")

mock_ = async_patcher.start()
print(mock_)

print("\nPatching async static method")
async_patcher = patch("__main__.Helper.async_static_method")
print(f"{Helper.async_static_method = }")
print(f"{Helper.__dict__['async_static_method'] = }")
print(f"{inspect.iscoroutinefunction(Helper.async_static_method) = }")
print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = }")
print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = }")

mock_ = async_patcher.start()
print(mock_)


$ python3.8 bpo39082.py
Patching async static method
Helper.async_class_method = <bound method Helper.async_class_method of <class '__main__.Helper'>>
Helper.__dict__['async_class_method'] = <classmethod object at 0x10de7bcd0>
inspect.iscoroutinefunction(Helper.async_class_method) = True
inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = False
inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = True
<MagicMock name='async_class_method' id='4537489440'>

Patching async static method
Helper.async_static_method = <function Helper.async_static_method at 0x10e77baf0>
Helper.__dict__['async_static_method'] = <staticmethod object at 0x10de7bbb0>
inspect.iscoroutinefunction(Helper.async_static_method) = True
inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = False
inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = True
<MagicMock name='async_static_method' id='4537741520'>


Detect __func__ to be used for iscoroutinefunction

diff --git Lib/unittest/mock.py Lib/unittest/mock.py
index cd5a2aeb60..572468ca8d 100644
--- Lib/unittest/mock.py
+++ Lib/unittest/mock.py
@@ -48,6 +48,8 @@ _safe_super = super
 def _is_async_obj(obj):
     if _is_instance_mock(obj) and not isinstance(obj, AsyncMock):
         return False
+    if hasattr(obj, '__func__'):
+        obj = getattr(obj, '__func__')
     return asyncio.iscoroutinefunction(obj) or inspect.isawaitable(obj)

[0] https://github.com/python/cpython/blob/50d4f12958bf806a4e1a1021d70cfd5d448c5cba/Lib/unittest/mock.py#L1387
[1] https://github.com/python/cpython/blob/50d4f12958bf806a4e1a1021d70cfd5d448c5cba/Lib/unittest/mock.py#L48
msg358618 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-18 09:21
Found Raymond's answer on the difference between __dict__ and attribute lookup regarding descriptors to be useful here : https://stackoverflow.com/a/44600603
msg360470 - (view) Author: Matt Kokotovich (mkokotovich) * Date: 2020-01-22 14:39
I'd love to see this issue resolved, as it is keeping me from being able to switch to 3.8. 

I have a PR with Karthikeyan's suggestion, as I agree it makes more sense and could apply to more cases: https://github.com/python/cpython/pull/18116
msg360679 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2020-01-25 10:17
New changeset 62865f4532094017a9b780b704686ca9734bc329 by Chris Withers (Matthew Kokotovich) in branch 'master':
bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18116)
https://github.com/python/cpython/commit/62865f4532094017a9b780b704686ca9734bc329
msg360727 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2020-01-26 15:30
New changeset 19be85c76503535c101b38194d282187de0ff631 by Chris Withers (Matthew Kokotovich) in branch '3.8':
[3.8] bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18190)
https://github.com/python/cpython/commit/19be85c76503535c101b38194d282187de0ff631
History
Date User Action Args
2020-01-26 15:30:33cjw296setmessages: + msg360727
2020-01-26 02:33:44mkokotovichsetpull_requests: + pull_request17573
2020-01-25 10:18:45cjw296setstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-01-25 10:17:50cjw296setnosy: + cjw296
messages: + msg360679
2020-01-22 14:39:26mkokotovichsetmessages: + msg360470
2020-01-22 14:36:43python-devsetpull_requests: + pull_request17503
2020-01-20 20:45:40mkokotovichsetnosy: + mkokotovich
2019-12-27 20:31:19czardozsetkeywords: + patch
stage: patch review
pull_requests: + pull_request17162
2019-12-18 09:21:58xtreaksetmessages: + msg358618
2019-12-18 05:52:11xtreaksetmessages: + msg358612
components: + Library (Lib), - Tests, asyncio
2019-12-18 03:21:39xtreaksetnosy: + lisroach, xtreak
messages: + msg358609
2019-12-18 00:36:45czardozcreate