classification
Title: multiprocessing Manager mutable type member access failure
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: davin, vilnis.termanis
Priority: normal Keywords:

Created on 2016-10-12 13:03 by vilnis.termanis, last changed 2016-10-13 21:46 by ned.deily.

Files
File name Uploaded Description Edit
manager_pickling.py vilnis.termanis, 2016-10-12 13:03 Script demonstrating problem
Messages (1)
msg278529 - (view) Author: Vilnis Termanis (vilnis.termanis) * Date: 2016-10-12 13:03
Accessing some Manager types (e.g. Lock) through a list, dict or Namespace proxy is not possible if both the mutable container (e.g. list) and contained type instance (e.g. Lock) have been created in the same process.

Symptoms:

In accessing process:
multiprocessing.managers.RemoteError on access, e.g.:
Unserializable message: ('#RETURN', <thread.lock object at 0x7fcf40d31370>)

.. and in underlying manager:
_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

The provided test script performs:
0) Initialise SyncManager (via multiprocessing.Manager())
1) Create list proxy through manager
2) Insert another proxied type into the list (e.g. Lock)
3) Try to access type instance from (2) via container created in (1)

Note: When step (2) is run in a child process, everything work as expected. When all steps execute in the same process, one gets the aforementioned exception.


See also:
https://mail.python.org/pipermail/python-list/2009-September/552988.html
History
Date User Action Args
2016-10-13 21:46:41ned.deilysetnosy: + davin
2016-10-12 13:05:06vilnis.termanissettype: crash -> behavior
2016-10-12 13:04:47vilnis.termanissettype: crash
2016-10-12 13:03:47vilnis.termaniscreate