Title: Adding a SyncManager Queue proxy to a SyncManager dict or Namespace proxy raises an exception
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.7, Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Alexander Prokhorov, Locane, davin, finefoot, hniksic, jjdmon, pitrou, xiang.zhang
Priority: normal Keywords: patch

Created on 2017-05-03 11:53 by jjdmon, last changed 2019-09-24 00:52 by finefoot.

Pull Requests
URL Status Linked Edit
PR 4819 open uspike, 2017-12-12 20:02
PR 16341 open finefoot, 2019-09-24 00:52
Messages (5)
msg292889 - (view) Author: Jeff DuMonthier (jjdmon) Date: 2017-05-03 11:53
In multiprocessing, attempting to add a Queue proxy to a dict or Namespace proxy (all returned by the same SyncManager) raises an exception indicating a keyword argument 'manager_owned=True' has been passed to the function AutoProxy() but is not an argument of that function.

In lib/python3.6/multiprocessing/, in function RebuildProxy(), line 873: "kwds['manager_owned'] = True" adds this argument to a keyword argument dictionary.  This function calls AutoProxy which has an argument list defined on lines 909-910 as:
def AutoProxy(token, serializer, manager=None, authkey=None,
              exposed=None, incref=True):
This raises an exception because there is no manager_owned argument defined.  I added "manager_owned=False" as a keyword argument to AutoProxy which seems to have fixed the problem.  There is no exception and I am able to pass Queue proxies through dict and Namespace proxies to other processes and use them.  I don't know the purpose of that argument though or if the AutoProxy function should actually use it for something.  My fix allows but ignores it.
msg311592 - (view) Author: Hrvoje Nikšić (hniksic) * Date: 2018-02-04 08:40
I encountered this bug while testing the code in this StackOverflow answer:

The code at the end of the answer runs on Python 3.5, but fails on 3.6 with the "unexpected keyword argument 'manager_owned'" error.

If someone knows of a workaround until the PR is accepted, it would be appreciated as well.
msg311594 - (view) Author: Hrvoje Nikšić (hniksic) * Date: 2018-02-04 10:16
The issue is also present in Python 3.7.0b1.
msg346334 - (view) Author: (finefoot) * Date: 2019-06-23 19:10
This is still an issue:

Is there a specific reason, why doesn't get reviewed?
msg352960 - (view) Author: Michael Tippie (Locane) Date: 2019-09-22 05:50
I am getting this error now, too.  I'm not sure what's causing it - when I use multiprocessing with a Manager.Queue, if I am passing around an object on the queue stack, I get this error.

Would really like it to "just work".
Date User Action Args
2019-09-24 00:52:11finefootsetpull_requests: + pull_request15918
2019-09-22 05:50:49Locanesetnosy: + Locane
messages: + msg352960
2019-06-23 19:10:54finefootsetnosy: + finefoot
messages: + msg346334
2018-07-02 12:02:53xiang.zhangsetnosy: + pitrou, davin, xiang.zhang
2018-02-04 10:16:59hniksicsetmessages: + msg311594
versions: + Python 3.7
2018-02-04 08:40:20hniksicsetnosy: + hniksic
messages: + msg311592
2017-12-12 20:02:38uspikesetkeywords: + patch
stage: patch review
pull_requests: + pull_request4709
2017-11-27 22:11:39Alexander Prokhorovsetnosy: + Alexander Prokhorov
2017-05-03 11:53:34jjdmoncreate