Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AsyncMock is unable to correctly patch static or class methods #83263

Closed
czardoz mannequin opened this issue Dec 18, 2019 · 7 comments
Closed

AsyncMock is unable to correctly patch static or class methods #83263

czardoz mannequin opened this issue Dec 18, 2019 · 7 comments
Labels
3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir

Comments

@czardoz
Copy link
Mannequin

czardoz mannequin commented Dec 18, 2019

BPO 39082
Nosy @cjw296, @asvetlov, @1st1, @czardoz, @lisroach, @tirkarthi, @mkokotovich
PRs
  • bpo-39082: Fix: AsyncMock is unable to correctly patch static/class methods #17717
  • bpo-39082: Allow AsyncMock to correctly patch static/class methods #18116
  • [3.8] bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18116) #18190
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-01-25.10:18:45.503>
    created_at = <Date 2019-12-18.00:36:45.699>
    labels = ['3.8', 'library', '3.9']
    title = 'AsyncMock is unable to correctly patch static or class methods'
    updated_at = <Date 2020-01-26.15:30:33.515>
    user = 'https://github.com/czardoz'

    bugs.python.org fields:

    activity = <Date 2020-01-26.15:30:33.515>
    actor = 'cjw296'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-01-25.10:18:45.503>
    closer = 'cjw296'
    components = ['Library (Lib)']
    creation = <Date 2019-12-18.00:36:45.699>
    creator = 'czardoz'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 39082
    keywords = ['patch']
    message_count = 7.0
    messages = ['358601', '358609', '358612', '358618', '360470', '360679', '360727']
    nosy_count = 7.0
    nosy_names = ['cjw296', 'asvetlov', 'yselivanov', 'czardoz', 'lisroach', 'xtreak', 'mkokotovich']
    pr_nums = ['17717', '18116', '18190']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue39082'
    versions = ['Python 3.8', 'Python 3.9']

    @czardoz
    Copy link
    Mannequin Author

    czardoz mannequin commented Dec 18, 2019

    Currently, patch is unable to correctly patch coroutinefunctions decorated with @staticmethod or @classmethod.

    Example:

    [*] aniketpanse [~/git/cpython] -> ./python                                                                                                                                                                                                         ±[master]
    Python 3.9.0a1+ (heads/master:50d4f12958, Dec 17 2019, 16:31:30) 
    [GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> class Helper:
    ...     @classmethod
    ...     async def async_class_method(cls):
    ...         pass
    ... 
    >>> from unittest.mock import patch
    >>> patch("Helper.async_class_method")
    <unittest.mock._patch object at 0x7fc28ddbbf40>
    

    This should ideally return an AsyncMock().

    @czardoz czardoz mannequin added tests Tests in the Lib/test dir topic-asyncio 3.8 only security fixes 3.9 only security fixes labels Dec 18, 2019
    @tirkarthi
    Copy link
    Member

    Using patch as a function just returns a patch object. You need to call start method on the patch object to return a Mock.

    @tirkarthi
    Copy link
    Member

    There seems to be a difference in obtaining the attribute out of the __dict__ and getattr. patch uses __dict__ [0] to access the attribute to be patched and falls back to getattr. There is a difference in detecting normal attribute access to be a coroutine function versus the one obtained from __dict__ . We can perhaps make _is_async_obj [1] to see if the passed obj is a classmethod/staticmethod and to use __func__ to detect a coroutine function.

    As a workaround for now you can pass AsyncMock explicitly to patch to return an AsyncMock. There was an open issue (bpo-36092) about patch and staticmethods/classmethods but not sure this is related to this.

    Retagging it to remove asyncio. Thanks for the report!

    # bpo-39082.py

    from unittest.mock import patch
    import inspect
    
    class Helper:
    @classmethod
    async def async_class_method(cls):
        pass
    
    @staticmethod
    async def async_static_method(*args, **kwargs):
        pass
    
    print("Patching async static method")
    async_patcher = patch("__main__.Helper.async_class_method")
    print(f"{Helper.async_class_method = }")
    print(f"{Helper.__dict__['async_class_method'] = }")
    print(f"{inspect.iscoroutinefunction(Helper.async_class_method) = }")
    print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = }")
    print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = }")
    
    mock_ = async_patcher.start()
    print(mock_)
    
    print("\nPatching async static method")
    async_patcher = patch("__main__.Helper.async_static_method")
    print(f"{Helper.async_static_method = }")
    print(f"{Helper.__dict__['async_static_method'] = }")
    print(f"{inspect.iscoroutinefunction(Helper.async_static_method) = }")
    print(f"{inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = }")
    print(f"{inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = }")
    
    mock_ = async_patcher.start()
    print(mock_)
    $ python3.8 bpo39082.py
    Patching async static method
    Helper.async_class_method = <bound method Helper.async_class_method of <class '__main__.Helper'>>
    Helper.__dict__['async_class_method'] = <classmethod object at 0x10de7bcd0>
    inspect.iscoroutinefunction(Helper.async_class_method) = True
    inspect.iscoroutinefunction(Helper.__dict__['async_class_method']) = False
    inspect.iscoroutinefunction(getattr(Helper, 'async_class_method')) = True
    <MagicMock name='async_class_method' id='4537489440'>

    Patching async static method
    Helper.async_static_method = <function Helper.async_static_method at 0x10e77baf0>
    Helper.__dict__['async_static_method'] = <staticmethod object at 0x10de7bbb0>
    inspect.iscoroutinefunction(Helper.async_static_method) = True
    inspect.iscoroutinefunction(Helper.__dict__['async_static_method']) = False
    inspect.iscoroutinefunction(getattr(Helper, 'async_static_method')) = True
    <MagicMock name='async_static_method' id='4537741520'>

    Detect __func__ to be used for iscoroutinefunction

    diff --git Lib/unittest/mock.py Lib/unittest/mock.py
    index cd5a2aeb60..572468ca8d 100644
    --- Lib/unittest/mock.py
    +++ Lib/unittest/mock.py
    @@ -48,6 +48,8 @@ _safe_super = super
     def _is_async_obj(obj):
         if _is_instance_mock(obj) and not isinstance(obj, AsyncMock):
             return False
    +    if hasattr(obj, '__func__'):
    +        obj = getattr(obj, '__func__')
         return asyncio.iscoroutinefunction(obj) or inspect.isawaitable(obj)

    [0]

    original = target.__dict__[name]

    [1]
    def _is_async_obj(obj):

    @tirkarthi tirkarthi added stdlib Python modules in the Lib dir and removed tests Tests in the Lib/test dir topic-asyncio labels Dec 18, 2019
    @tirkarthi
    Copy link
    Member

    Found Raymond's answer on the difference between __dict__ and attribute lookup regarding descriptors to be useful here : https://stackoverflow.com/a/44600603

    @mkokotovich
    Copy link
    Mannequin

    mkokotovich mannequin commented Jan 22, 2020

    I'd love to see this issue resolved, as it is keeping me from being able to switch to 3.8.

    I have a PR with Karthikeyan's suggestion, as I agree it makes more sense and could apply to more cases: #18116

    @cjw296
    Copy link
    Contributor

    cjw296 commented Jan 25, 2020

    New changeset 62865f4 by Chris Withers (Matthew Kokotovich) in branch 'master':
    bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18116)
    62865f4

    @cjw296 cjw296 closed this as completed Jan 25, 2020
    @cjw296 cjw296 closed this as completed Jan 25, 2020
    @cjw296
    Copy link
    Contributor

    cjw296 commented Jan 26, 2020

    New changeset 19be85c by Chris Withers (Matthew Kokotovich) in branch '3.8':
    [3.8] bpo-39082: Allow AsyncMock to correctly patch static/class methods (GH-18190)
    19be85c

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants