classification
Title: Multiprocessing Manager on 3D list - no change of the list possible
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: John_81, berker.peksag, davin, pitrou
Priority: normal Keywords:

Created on 2018-01-12 15:05 by John_81, last changed 2019-03-31 17:37 by berker.peksag. This issue is now closed.

Messages (2)
msg309858 - (view) Author: Johannes (John_81) Date: 2018-01-12 15:05
I have the following code, which works without multiprocessing:

data=[[['','','','','','','','','','','','']]]
data[0][0][0] = 5
data[0][0][1] = "5" # data in the array is mixed with float and str
print(data)

#=> [[[5, '5', '', '', '', '', '', '', '', '', '', '']]]

Now I want to use Multiprocessing and every process should be able to change the 3D list. This doesn't work and no error message is shown.

from multiprocessing import Process, Manager
manager=Manager()
data=manager.list([[['','','','','','','','','','','','']]])
data[0][0][0] = 5
data[0][0][1] = "5"
print(data)    

#=> [[['', '', '', '', '', '', '', '', '', '', '', '']]]

I found the following text:

list(sequence)
Create a shared list object and return a proxy for it.
Changed in version 3.6: Shared objects are capable of being nested. For     
example, a shared container object such as a shared list can contain 
other shared objects which will all be managed and synchronized by the 
SyncManager.(https://docs.python.org/3/library/multiprocessing.html)

Unfortunately it also doesn't work with 3.6.3, same problem as before! But as it should work, I guess it's a bug?

I use Ubuntu 16.04...
msg339273 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2019-03-31 17:37
I'm a bit late to reply, but I think you are using the API wrong. Please try the following snippet:

from multiprocessing import Manager

manager = Manager()

data = manager.list([manager.list([manager.list(['', '', '', '', '', '', '', '', '', '', '', ''])])])

print(data[0][0])
print(data[0][0][0])
print(data[0][0][1])

data[0][0][0] = 5
data[0][0][1] = '5'

print(data[0][0])
print(data[0][0][0])
print(data[0][0][1])

Output:

$ ./python.exe example.py
['', '', '', '', '', '', '', '', '', '', '', '']


[5, '5', '', '', '', '', '', '', '', '', '', '']
5
5

You can also take a look at the tests in the original commit that implemented this feature:

https://github.com/python/cpython/commit/86a76684269f940a20366cb42668f1acb0982dca
History
Date User Action Args
2019-03-31 17:37:27berker.peksagsetcomponents: + Library (Lib), - Interpreter Core
2019-03-31 17:37:10berker.peksagsetstatus: open -> closed

nosy: + pitrou, berker.peksag, davin
messages: + msg339273

resolution: not a bug
stage: resolved
2018-01-12 15:05:56John_81create