This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: mock.patch.stopall doesn't work with patch.dict
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: michael.foord Nosy List: Antoni Szych, cjw296, eric.snow, kakuma, kushal.das, lisroach, lwcolton, mariocj89, michael.foord, xtreak
Priority: normal Keywords: patch

Created on 2014-05-29 03:35 by kakuma, last changed 2022-04-11 14:58 by admin. This issue is now closed.

File name Uploaded Description Edit
add_stopall_patch_dict.patch kakuma, 2014-06-02 10:49 a patch to support patch.dict by patch.stopall.
support_patch_dict_by_stopall.diff kakuma, 2014-06-06 14:49
support_patch_dict_by_stopall.diff kakuma, 2014-06-07 16:34
Pull Requests
URL Status Linked Edit
PR 17606 merged mariocj89, 2019-12-14 14:42
Messages (22)
msg219331 - (view) Author: fumihiko kakuma (kakuma) * Date: 2014-05-29 03:35
It seems that stopall doesn't work when do start patch.dict to sys.modules.
I show sample scripts the following.

Using stopall test case seems to always refer the first mock foo object.
But using stop it refers a new mock foo object.

$ cat
import foo

def myfunc():
    print "myfunc foo=%s" % foo
    return foo
$ cat
import mock
import sys
import unittest

class SampTestCase(unittest.TestCase):
    def setUp(self):
        self.foo_mod = mock.Mock()
        self.m = mock.patch.dict('sys.modules', {'foo': self.foo_mod})
        self.p = self.m.start()
        print "foo_mod=%s" % self.foo_mod
        self.test_sampmod = sys.modules['test_sampmod']

    def tearDown(self):
        if len(sys.argv) > 1:
            print ">>> stop patch"
            print ">>> stopall patch"

    def test_samp1(self):
        self.assertEqual(self.foo_mod, self.test_sampmod.myfunc())

    def test_samp2(self):
        self.assertEqual(self.foo_mod, self.test_sampmod.myfunc())

    def test_samp3(self):
        self.assertEqual(self.foo_mod, self.test_sampmod.myfunc())

if __name__ == '__main__':
    suite = unittest.TestSuite()
$ python stop
test_samp1 (__main__.SampTestCase) ... foo_mod=<Mock id='41504336'>
myfunc foo=<Mock id='41504336'>
>>> stop patch
test_samp2 (__main__.SampTestCase) ... foo_mod=<Mock id='41504464'>
myfunc foo=<Mock id='41504464'>
>>> stop patch
test_samp3 (__main__.SampTestCase) ... foo_mod=<Mock id='41504720'>
myfunc foo=<Mock id='41504720'>
>>> stop patch

Ran 3 tests in 0.004s

$ python
test_samp1 (__main__.SampTestCase) ... foo_mod=<Mock id='19152464'>
myfunc foo=<Mock id='19152464'>
>>> stopall patch
test_samp2 (__main__.SampTestCase) ... foo_mod=<Mock id='19152592'>
myfunc foo=<Mock id='19152464'>
>>> stopall patch
test_samp3 (__main__.SampTestCase) ... foo_mod=<Mock id='19182096'>
myfunc foo=<Mock id='19152464'>
>>> stopall patch

FAIL: test_samp2 (__main__.SampTestCase)
Traceback (most recent call last):
  File "", line 27, in test_samp2
    self.assertEqual(self.foo_mod, self.test_sampmod.myfunc())
AssertionError: <Mock id='19152592'> != <Mock id='19152464'>

FAIL: test_samp3 (__main__.SampTestCase)
Traceback (most recent call last):
  File "", line 30, in test_samp3
    self.assertEqual(self.foo_mod, self.test_sampmod.myfunc())
AssertionError: <Mock id='19182096'> != <Mock id='19152464'>

Ran 3 tests in 0.003s

FAILED (failures=2)
msg219567 - (view) Author: fumihiko kakuma (kakuma) * Date: 2014-06-02 10:49
Checking 1.0.1) it seems that patch.stopall does not support patch.dict. Does it have any problem to support ptch.dict by stopall.
I made the attached patch file for this. It seems to work well. How about this?

But I don't know why sys.modules refers the first mock object. 

$ python
test_samp1 (__main__.SampTestCase) ... foo_mod=<Mock id='140164117280080'>
myfunc foo=<Mock id='140164117280080'>
>>> stopall patch
test_samp2 (__main__.SampTestCase) ... foo_mod=<Mock id='140164117280208'>
myfunc foo=<Mock id='140164117280208'>
>>> stopall patch
test_samp3 (__main__.SampTestCase) ... foo_mod=<Mock id='140164117280464'>
myfunc foo=<Mock id='140164117280464'>
>>> stopall patch

Ran 3 tests in 0.001s

msg219697 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-06-03 15:15
Yep, patch.dict wasn't designed with stopall in mind so it needs adding. Thanks for pointing this out and your fix. 

Your patch isn't quite right, those operations shouldn't be inside the try excepts. (And there are no tests.)
msg219885 - (view) Author: fumihiko kakuma (kakuma) * Date: 2014-06-06 14:49
Thank you for your reply.
Yes, you are right. The patch was too slapdash. I re-created it and added unit tests.
msg219887 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-06-06 14:53
That's better - thanks. Another minor tweak needed though. stopall should only stop patches that were started with "start", not those used as context managers or decorators (or they will be stopped twice!). 

See how the main patch object only adds to the set of active patches in the start method, not in __enter__ (and removes in stop rather than __exit__).
msg219951 - (view) Author: fumihiko kakuma (kakuma) * Date: 2014-06-07 16:34
Hi michael,
Certainly, thank you for your many advices. I attached the new patch file.
msg220030 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-06-08 12:05
That looks great - thanks! I'll get it committed shortly.
msg220082 - (view) Author: fumihiko kakuma (kakuma) * Date: 2014-06-09 04:47
Thank you in advance.
msg229291 - (view) Author: Colton Leekley-Winslow (lwcolton) * Date: 2014-10-14 14:37
What's the status on this?
msg229292 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-10-14 14:38
It just needs committing, I believe Kushal Das has volunteered to do it.
msg229294 - (view) Author: Kushal Das (kushal.das) * (Python committer) Date: 2014-10-14 14:40
I will do that. New job is taking time.
msg232411 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-12-10 03:35
FYI, you probably don't want to be patching out sys.modules.  It isn't guaranteed that doing so will have the expected effect on import semantics.
msg232435 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-12-10 16:57
why not?
msg232440 - (view) Author: Colton Leekley-Winslow (lwcolton) * Date: 2014-12-10 19:11
Patching sys.modules is an idea I got  from the official documentation
msg232458 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-12-11 03:52
At least for CPython sys.modules is initially set to the modules dict on the interpreter struct.  As of 3.4 the import system itself only cares about sys.modules (I'll have to double check on builtin___import__).  However, if I recall correctly at least part of the import C-API interacts directly with the original interpreter copy of the modules dict.

The catch is that setting sys.modules to something else does not also change the dict on the interpreter struct.  So they will be out of sync.  This may cause problems.  In practice I don't think it's a big deal, but there is no guarantee that patching out sys.modules will give you the behavior that you expect.

See also issue #12633.
msg232467 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-12-11 10:18
Using patch.dict manipulates the contents of sys.modules, it doesn't replace sys.modules.
msg232480 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2014-12-11 14:57
Ah.  Never mind then. :)
msg298203 - (view) Author: Antoni Szych (Antoni Szych) Date: 2017-07-12 11:44

It's been 3 years now since this issue was first raised.
We bumped upon this issue while using code like following:

def tearDown():

def test123():
    assert False

While `patch.stopall()` is run, it doesn't stop anything. This is because `p.start()` in fact executes `mock._patch_dict._patch_dict()`, which does not execute `self._active_patches.append(self)` (like ordinary `p=patch(...).start()` would).

I could understand that this is just a design choice (which may seem unintuitive for me, but possibly perfectly good for others), however the official documentation [1] states:

>**All** the patchers have start() and stop() methods.

And few lines below [2], we have:

>patch.stopall() Stop all active patches. Only stops patches started with start.

The above is not true for `patch.dict()`, so the documentation is unfortunatelly misleading.

Is there a possibility to fix this in some 3.5 maintenance release (preferably code, not docs :) )?

If anyone else will have the same issue: for now a workaround would be to use `patch.dict()` either as a decorator, or as a context manager (`with patch.dict()`).

msg322016 - (view) Author: Antoni Szych (Antoni Szych) Date: 2018-07-20 15:26

Any chance of getting this resolved 1 year later? :)

msg358323 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-13 09:05
This sounds like a good idea given the docs and the general agreement in the thread. I would like to target this for 3.9 . Differentiation between the normal patches and the ones from patch.dict is nice but given that docstring indicates LIFO for all active patches storing patch.dict items also in _patch.active_patches ensures we can do LIFO if there is a mixture of normal and dictionary patches started with start method. Adding other maintainers for thoughts.

diff --git Lib/unittest/ Lib/unittest/
index cd5a2aeb60..96115e06ba 100644
--- Lib/unittest/
+++ Lib/unittest/
@@ -1853,8 +1853,21 @@ class _patch_dict(object):
         return False

-    start = __enter__
-    stop = __exit__
+    def start(self):
+        """Activate a patch, returning any created mock."""
+        result = self.__enter__()
+        _patch._active_patches.append(self)
+        return result
+    def stop(self):
+        """Stop an active patch."""
+        try:
+            _patch._active_patches.remove(self)
+        except ValueError:
+            # If the patch hasn't been started this will fail
+            pass
+        return self.__exit__()
msg358324 - (view) Author: Mario Corchero (mariocj89) * (Python triager) Date: 2019-12-13 09:57
Makes total sense, I think we should get this for 3.9.
Not sure I'll backport this, even if a bugfix it might cause unexpected changes (Though I'd be OK with it, given that calling stop twice causes no issue).

I'm happy to push a PR with the proposed change and some tests if you want @xtreak. This is quite a simple fix we can get through in a short time.
msg358374 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-12-14 04:40
Thanks Mario. I think the last patch attached to the issue is complete with tests and needs to be updated master. Feel free to raise a PR.
Date User Action Args
2022-04-11 14:58:04adminsetgithub: 65799
2020-01-24 08:41:44cjw296setstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-12-14 14:42:34mariocj89setstage: test needed -> patch review
pull_requests: + pull_request17075
2019-12-14 04:40:21xtreaksetmessages: + msg358374
2019-12-13 09:57:20mariocj89setmessages: + msg358324
2019-12-13 09:05:59xtreaksetversions: + Python 3.9, - Python 3.5
nosy: + lisroach, xtreak, cjw296, mariocj89

messages: + msg358323

components: + Library (Lib), - Tests
2018-07-21 02:36:14ppperrysettitle: mock.patch.stopall doesn't work with patch.dict to sys.modules -> mock.patch.stopall doesn't work with patch.dict
2018-07-20 15:26:23Antoni Szychsetmessages: + msg322016
2017-07-12 11:44:02Antoni Szychsetnosy: + Antoni Szych
messages: + msg298203
2014-12-11 14:57:10eric.snowsetmessages: + msg232480
2014-12-11 10:18:07michael.foordsetmessages: + msg232467
2014-12-11 03:52:13eric.snowsetmessages: + msg232458
2014-12-10 19:11:38lwcoltonsetmessages: + msg232440
2014-12-10 16:57:58michael.foordsetmessages: + msg232435
2014-12-10 03:35:58eric.snowsetnosy: + eric.snow
messages: + msg232411
2014-10-14 14:40:40kushal.dassetmessages: + msg229294
2014-10-14 14:38:00michael.foordsetmessages: + msg229292
2014-10-14 14:37:15lwcoltonsetnosy: + lwcolton
messages: + msg229291
2014-06-09 04:47:42kakumasetmessages: + msg220082
2014-06-08 12:05:59michael.foordsetmessages: + msg220030
2014-06-07 16:34:09kakumasetfiles: + support_patch_dict_by_stopall.diff

messages: + msg219951
2014-06-06 14:53:13michael.foordsetmessages: + msg219887
2014-06-06 14:49:01kakumasetfiles: + support_patch_dict_by_stopall.diff

messages: + msg219885
2014-06-03 15:15:14michael.foordsetassignee: michael.foord
stage: test needed
messages: + msg219697
versions: + Python 3.5, - Python 2.7
2014-06-03 14:47:37r.david.murraysetnosy: + kushal.das
2014-06-02 10:49:55kakumasetfiles: + add_stopall_patch_dict.patch
keywords: + patch
messages: + msg219567
2014-05-29 18:16:32ned.deilysetnosy: + michael.foord
2014-05-29 03:35:28kakumacreate