New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
multiprocessing.managers.Server: problem with returning proxy of registered object #65053
Comments
class MyManager(BaseManager): pass
MyManager.register('MyClass', MyClass, method_to_typeid={'get_child':'MyClass'})
373 def create(self, c, typeid, *args, **kwds): This means that if ANY method registered with a Manager should return a proxy for a registered typeid, for which a callable is provided, it will fail, unless the callable is capable to handle such unexpected arguments. |
I am not sure method_to_typeid and create_method were really intended to be public -- they are only used by Pool proxies. You can maybe work around the problem by registering a second typeid without specifying callable. That can be used in method_to_typeid: import multiprocessing.managers
class MyClass(object):
def __init__(self):
self._children = {}
def get_child(self, i):
return self._children.setdefault(i, type(self)())
def __repr__(self):
return '<MyClass %r>' % self._children
class MyManager(multiprocessing.managers.BaseManager):
pass
MyManager.register('MyClass', MyClass,
method_to_typeid = {'get_child': '_MyClass'})
MyManager.register('_MyClass',
method_to_typeid = {'get_child': '_MyClass'},
create_method=False)
if __name__ == '__main__':
m = MyManager()
m.start()
try:
a = m.MyClass()
b = a.get_child(1)
c = b.get_child(2)
d = c.get_child(3)
print a # <MyClass {1: <MyClass {2: <MyClass {3: <MyClass {}>}>}>}>
finally:
m.shutdown() |
Thanks for the suggestion. Another issue that arises in my case is: when I try to pass a proxy of MyClass to a subprocess it looses its' _manager during pickling and thus the ability to create proxies for children returned by get_child. This is solved by reimplementing the (not-working: http://bugs.python.org/issue5862) __reduce__ method of BaseManager in MyManager and creating corresponding custom proxy for MyClass with __reduce__ method also reimplemented. So the working solution for the situation is: either class MyClass(object):
__metaclass__ = ReturnProxy
###class body### or class AutoProxyMeta(type): class MyClassProxy(BaseProxy):
class MyClassManager(UManager):
def __reduce__(self):
return (RebuildMyClassManager,
(self._address, None, self._serializer))
WorkCounterManager.register('MyClass', MyClass, MyClassProxy)
#optionally: WorkCounterManager.register('_MyClass', None, MyClassProxy, create_method=False)
def RebuildMyClassManager(address, authkey, serializer):
mgr = MyClassManager(address, authkey, serializer)
mgr.connect()
return mgr |
It appears that the multiple workarounds proposed by the OP (@allista) address the original request and that there is no bug or unintended behavior arising from multiprocessing itself. Combined with the lack of activity in this discussion, I'm inclined to believe that the workarounds have satisfied the OP and this issue should be closed. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: