classification
Title: Adding a SyncManager Queue proxy to a SyncManager dict or Namespace proxy raises an exception
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alexander Prokhorov, davin, finefoot, hniksic, jjdmon, pitrou, xiang.zhang
Priority: normal Keywords: patch

Created on 2017-05-03 11:53 by jjdmon, last changed 2019-06-23 19:10 by finefoot.

Pull Requests
URL Status Linked Edit
PR 4819 open uspike, 2017-12-12 20:02
Messages (4)
msg292889 - (view) Author: Jeff DuMonthier (jjdmon) Date: 2017-05-03 11:53
In multiprocessing, attempting to add a Queue proxy to a dict or Namespace proxy (all returned by the same SyncManager) raises an exception indicating a keyword argument 'manager_owned=True' has been passed to the function AutoProxy() but is not an argument of that function.

In lib/python3.6/multiprocessing/managers.py, in function RebuildProxy(), line 873: "kwds['manager_owned'] = True" adds this argument to a keyword argument dictionary.  This function calls AutoProxy which has an argument list defined on lines 909-910 as:
def AutoProxy(token, serializer, manager=None, authkey=None,
              exposed=None, incref=True):
This raises an exception because there is no manager_owned argument defined.  I added "manager_owned=False" as a keyword argument to AutoProxy which seems to have fixed the problem.  There is no exception and I am able to pass Queue proxies through dict and Namespace proxies to other processes and use them.  I don't know the purpose of that argument though or if the AutoProxy function should actually use it for something.  My fix allows but ignores it.
msg311592 - (view) Author: Hrvoje Nikšić (hniksic) * Date: 2018-02-04 08:40
I encountered this bug while testing the code in this StackOverflow answer:

    https://stackoverflow.com/a/48565011/1600898

The code at the end of the answer runs on Python 3.5, but fails on 3.6 with the "unexpected keyword argument 'manager_owned'" error.

If someone knows of a workaround until the PR is accepted, it would be appreciated as well.
msg311594 - (view) Author: Hrvoje Nikšić (hniksic) * Date: 2018-02-04 10:16
The issue is also present in Python 3.7.0b1.
msg346334 - (view) Author: (finefoot) Date: 2019-06-23 19:10
This is still an issue: https://stackoverflow.com/questions/56716470/python-multiprocessing-nested-shared-objects-doesnt-work-with-queue

Is there a specific reason, why https://github.com/python/cpython/pull/4819 doesn't get reviewed?
History
Date User Action Args
2019-06-23 19:10:54finefootsetnosy: + finefoot
messages: + msg346334
2018-07-02 12:02:53xiang.zhangsetnosy: + pitrou, davin, xiang.zhang
2018-02-04 10:16:59hniksicsetmessages: + msg311594
versions: + Python 3.7
2018-02-04 08:40:20hniksicsetnosy: + hniksic
messages: + msg311592
2017-12-12 20:02:38uspikesetkeywords: + patch
stage: patch review
pull_requests: + pull_request4709
2017-11-27 22:11:39Alexander Prokhorovsetnosy: + Alexander Prokhorov
2017-05-03 11:53:34jjdmoncreate