This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Mock.assert_has_calls works strange
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.10, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: cjw296, corona10, dmitriy.mironiyk, kamilturek, lisroach, mardoxx, mariocj89, michael.foord, roysmith, xtreak
Priority: normal Keywords:

Created on 2021-03-02 14:31 by dmitriy.mironiyk, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
test_mock_asser_has_calls.py dmitriy.mironiyk, 2021-03-02 14:31
Messages (4)
msg387932 - (view) Author: Dmitriy Mironiyk (dmitriy.mironiyk) Date: 2021-03-02 14:31
I think that behavior of Mock.assert_has_calls is misleading. 
- When I call Mock.assert_has_calls with any_order=False it checks that expected calls are the same calls as performed on mock and raise an error if mock has some calls other than expected.
- When I call Mock.assert_has_calls with any_order=True it checks that mock has expected calls and not raise an error when mock has other calls than expected.

I suppose should be two separate methods:
- Mock.assert_has_only_calls that always raise an error when mock has calls other than expected(not regarding any_order).
- Mock.assert_has_calls that raise error only when no calls that provided in expected or if order of calls is wrong when any_order=False.
msg388758 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2021-03-15 18:47
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls

> If any_order is false then the calls must be sequential. There can be extra calls before or after the specified calls.

One way to check that the calls appear in order even with extra calls in between since any_order=False expects the calls to be sequential is to use mock_calls which is a list and use index method. It's not very elegant.

>>> from unittest.mock import Mock, call
>>> m = Mock()
>>> m(1)
<Mock name='mock()' id='139845749964176'>
>>> m(2)
<Mock name='mock()' id='139845749964176'>
>>> m(3)
<Mock name='mock()' id='139845749964176'> 
>>> m.mock_calls.index(call(1))
0
>>> m.mock_calls.index(call(3))
2

> - Mock.assert_has_only_calls that always raise an error when mock has calls other than expected(not regarding any_order).

This sounds like a stricter version of assert_has_calls to ensure expected is matched without order with mock_calls and len(expected) == len(mock_calls)

I am not keen on changing behavior of assert_has_calls with any additional flags for compatibility but any additional helper should go in python 3.10 only. So I have updated the version.
msg397172 - (view) Author: Mardoxx (mardoxx) Date: 2021-07-08 23:29
This is surprising behaviour, Python 3.8.9:

factory = unittest.mock.Mock()

### test example
foo_obj = factory.create("foo"),do_thing()

# !! MUST BE CALLED AFTER .create("foo") !!
bar_obj = factory.create("bar").do_thing()
###

# I set any_order to false because these should be called in order.
factory.create.assert_has_calls([
    unittest.mock.call("foo"),
    unittest.mock.call("bar")
], any_order = False)

# OOPS!! this fails
# mock_calls == [call('foo'), call().do_thing(), call('bar'), call().do_thing()]
# I can however check this with...
# assert factory.create.call_args_list == [
#     unittest.mock.call("foo"),
#     unittest.mock.call("bar")
# ]

Am I fundamentally misunderstanding something here?

This appears to be different behaviour from assert_has_awaits, which adds to the confusion. assert_has_calls checks mock_calls, I'd have thought it should check call_args_list. Another method assert_mock_calls should check mock_calls for consistency? Or have I missed something here?

Perhaps an additional flag of in_sequence would be nice, defaulted to False to keep the existing unusual behaviour.
When False, calls must be in exactly the order they are described.
When True, calls must be in sequence (with anything in-between each call).
any_order=True overrides this and allows them to be in any order anywhere in the mock_calls list.
msg410667 - (view) Author: Roy Smith (roysmith) Date: 2022-01-16 03:21
I agree that this is confusing and that what we need is an assertion for the top-level mock having specific calls in a specific order, and ignores any intervening extra calls to mocked functions.  In other words, a version of assert_has_calls() which looks at call_args_list instead of mock_calls.

I just finished up a session of head-banging with some tests that were failing (Python 3.7), and eventually ended up with the

self.assertEqual(my_mock.call_args_list, [call(...), call(...)])

idiom as noted in msg397172 (but without first banging a few new dents into the top of my desk).  This exact same experience is related in a recent stackoverflow thread (https://stackoverflow.com/questions/69360318/python-unittest-mock-assert-has-calls-returning-calls-to-other-mocks) so this seems to be a common source of confusion.

I am neutral on whether this is implemented as a new flag to assert_has_calls() or as a new assertion method.

As an aside, what I was trying to do was test if my code constructed its several instances of a class in the correct way.  At one point I hit upon the idea of:

MyMockedClass().__init__.assert_has_calls(....)

which expressed my desired logic exactly and simply, but couldn't get that to work.  It's unclear if I just never found the proper incantation, or if that's fundamentally unworkable.
History
Date User Action Args
2022-04-11 14:59:42adminsetgithub: 87537
2022-01-16 03:21:02roysmithsetnosy: + roysmith
messages: + msg410667
2021-07-08 23:29:43mardoxxsetnosy: + mardoxx

messages: + msg397172
versions: + Python 3.8
2021-03-15 18:47:49xtreaksetversions: + Python 3.10, - Python 3.7
nosy: + cjw296, michael.foord, lisroach, mariocj89

messages: + msg388758

components: + Library (Lib), - Tests
2021-03-03 21:19:04kamiltureksetnosy: + kamilturek
2021-03-03 04:07:07corona10setnosy: + corona10
2021-03-03 03:14:51xtreaksetnosy: + xtreak
2021-03-02 14:31:34dmitriy.mironiykcreate