classification
Title: Uninformative error message in multiprocessing.Manager()
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: davin Nosy List: berker.peksag, davin, python-dev, sbt, wojtekwalczak
Priority: normal Keywords: patch

Created on 2014-04-11 17:20 by wojtekwalczak, last changed 2016-09-08 19:53 by davin. This issue is now closed.

Files
File name Uploaded Description Edit
managers_tb_msg.patch wojtekwalczak, 2014-04-11 17:20 A patch review
Messages (7)
msg215934 - (view) Author: Wojciech Walczak (wojtekwalczak) * Date: 2014-04-11 17:20
While using multiprocessing.Manager() to send data between processes I've noticed that one of the traceback messages is not very informative:

Traceback (most recent call last):
  File "age_predict.py", line 39, in <module>
    train_data, train_target, test_data = prepare_data()
  File "age_predict.py", line 30, in prepare_data
    train_data = q.get(True, 10000)
  File "<string>", line 2, in get
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 777, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------
Unserializable message: ('#RETURN', [A WHOLE LOT OF DATA GOES HERE]
 

The real problem here is that my machine is running out of memory, but the traceback message shadows this information and reports "Unserializable message" instead.

After a simple path (see: attached file) the traceback message is as follows:


Traceback (most recent call last):
  File "age_predict.py", line 39, in <module>
    train_data, train_target, test_data = prepare_data()
  File "age_predict.py", line 30, in prepare_data
    train_data = q.get(True, 10000)
  File "<string>", line 2, in get
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 775, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/managers.py", line 288, in serve_client
    send(msg)
MemoryError: out of memory


Here's another example (see: http://bugs.python.org/issue6766):

This python3 code:

from multiprocessing import Manager
manager = Manager()
ns_proxy = manager.Namespace()
evt_proxy = manager.Event()
ns_proxy.my_event_proxy = evt_proxy
print(ns_proxy.my_event_proxy)


Produces following traceback message:


Traceback (most recent call last):
  File "test.py", line 6, in <module>
    print(ns_proxy.my_event_proxy)
  File "/cpython/Lib/multiprocessing/managers.py", line 1032, in __getattr__
    return callmethod('__getattribute__', (key,))
  File "/cpython/Lib/multiprocessing/managers.py", line 748, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
----------------------------------------------------------------------
Unserializable message: ('#RETURN', <threading.Event object at 0x7f9484aff278>)
----------------------------------------------------------------------


...and after applying the attached patch it becomes clearer what's going on:


Traceback (most recent call last):
  File "test.py", line 6, in <module>
    print(ns_proxy.my_event_proxy)
  File "/cpython/Lib/multiprocessing/managers.py", line 1032, in __getattr__
    return callmethod('__getattribute__', (key,))
  File "/cpython/Lib/multiprocessing/managers.py", line 748, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
----------------------------------------------------------------------
Unserializable message: Traceback (most recent call last):
  File "/cpython/Lib/multiprocessing/managers.py", line 276, in serve_client
    send(msg)
  File "/cpython/Lib/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/cpython/Lib/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

This patch doesn't break any tests.
msg236621 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2015-02-25 21:38
Can someone review the patch with a view to commit please.  It's a change to one line as explained in msg215934.
msg259110 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-01-28 09:21
Looks good to me. Perhaps we can use utils.info() to print the exception.
msg275070 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-09-08 17:17
Looks good to me as well.  I'll go ahead with applying the patch.

Berker:  I'm not sure it's worth adding something like `util.info("Serialization failure on: %r", msg)`.  If serialization failed for some reason other than MemoryError, we will be communicating the output of format_exc() back to the client.  If serialization failed due to a MemoryError (which has tripped up plenty of people) then our util.info may similarly fail.



Note that with the patch recently applied to issue6766, under 3.6 the second example no longer triggers an exception.  That second example can even continue:
    >>> ns_proxy.my_event_proxy.set()
    >>> ns_proxy.my_event_proxy.is_set()
    True
    >>> evt_proxy.is_set()
    True
    >>> evt_proxy.clear()
    >>> ns_proxy.my_event_proxy.is_set()
    False
msg275094 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-08 18:07
New changeset cbf81969aba4 by Davin Potts in branch '2.7':
Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions
https://hg.python.org/cpython/rev/cbf81969aba4
msg275126 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-08 19:47
New changeset 9b1f8c68de4c by Davin Potts in branch '3.5':
Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions
https://hg.python.org/cpython/rev/9b1f8c68de4c

New changeset 199aca18d9a1 by Davin Potts in branch 'default':
Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions
https://hg.python.org/cpython/rev/199aca18d9a1
msg275128 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2016-09-08 19:53
Thanks for your patch @wojtekwalczak!
History
Date User Action Args
2016-09-08 19:53:36davinsetstatus: open -> closed
resolution: fixed
messages: + msg275128

stage: patch review -> resolved
2016-09-08 19:47:49python-devsetmessages: + msg275126
2016-09-08 18:07:40python-devsetnosy: + python-dev
messages: + msg275094
2016-09-08 17:17:00davinsetmessages: + msg275070
2016-09-08 16:43:55davinsetassignee: davin
2016-01-28 16:48:46BreamoreBoysetnosy: - BreamoreBoy
2016-01-28 09:21:50berker.peksagsetversions: + Python 3.6, - Python 3.4
nosy: + berker.peksag, davin

messages: + msg259110

stage: patch review
2015-02-25 21:38:14BreamoreBoysetnosy: + BreamoreBoy
messages: + msg236621
2014-04-11 19:31:30ned.deilysetnosy: + sbt

versions: - Python 3.1, Python 3.2, Python 3.3
2014-04-11 17:20:20wojtekwalczakcreate