Title: multiprocessing.BaseManager does not retain Client type.
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: amol, davin, pitrou
Priority: normal Keywords: patch

Created on 2020-04-16 21:35 by amol, last changed 2020-04-19 19:43 by xtreak.

Pull Requests
URL Status Linked Edit
PR 19609 open amol, 2020-04-19 19:36
Messages (2)
msg366630 - (view) Author: Alessandro Molina (amol) * Date: 2020-04-16 21:35
When a new `multiprocessing.managers.BaseManager` instance is made, the client used to connect is not stable across the life of the object.

A very quick example to show that is 

from unittest.mock import Mock
from multiprocessing.managers import SyncManager, listener_client

listener_client["faked"] = listener_client["xmlrpclib"] 
sm = SyncManager(serializer="faked")

As expected the manager is created with the XmlClient

>>> print(sm._Client.__name__)

but in reality, if the "faked" serializer is changed in any way

listener_client["faked"] = (None, Mock(side_effect=RuntimeError("BROKEN")))

When trying to connect, we will unexpectedly connect with whatever it is the serializer registered at the time of connection instead of the one bound to the SyncManager instance 

>>> sm.connect()
Traceback (most recent call last):
  File "/home/amol/wrk/cpython/", line 17, in <module>
  File "/home/amol/wrk/cpython/Lib/multiprocessing/", line 521, in connect
    conn = Client(self._address, authkey=self._authkey)
  File "/home/amol/wrk/cpython/Lib/unittest/", line 1152, in _execute_mock_call
    raise effect
RuntimeError: BROKEN

To make things worse, we would actually randomly use XmlClient or new one depending on which SyncManager method we call.

This makes also inconvenient to replace the connection layer with a fake one during tests to simulate stub responses. Furthermore the client of the manager is also not propagated properly to the proxies created through that manager making even less consistent the behaviour.
msg366632 - (view) Author: Alessandro Molina (amol) * Date: 2020-04-16 21:50
The issue seems fairly easy to fix, it's just a matter in consistency in usage of self._Client attribute in the manager. I'm working on a PR that I'm willing to propose for review once I have finished it and wrote tests.
Date User Action Args
2020-04-19 19:43:21xtreaksetnosy: + pitrou, davin
2020-04-19 19:36:39amolsetkeywords: + patch
stage: patch review
pull_requests: + pull_request18942
2020-04-16 21:50:28amolsetmessages: + msg366632
2020-04-16 21:35:30amolcreate