➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing.Queue.put can fail silently due to pickle errors
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Sander Land
Priority: normal Keywords:

Created on 2020-04-05 10:55 by Sander Land, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
minimal_repr.py Sander Land, 2020-04-05 10:55 Minimal example for reproducing the behaviour
Messages (1)
msg365816 - (view) Author: Sander Land (Sander Land) Date: 2020-04-05 10:55 
The multiprocessing Queue uses a thread to pickle and send the object after a call to put. When pickling fails (e.g. due to recursion depth) the exception is not returned to the caller to .put but instead dumped on the screen, leaving any multiprocessing pools in a very unhappy state.

Suggested fix: pickle the object in the same thread as the caller and send the pickled object to the thread, ensuring the caller to .put can catch the exception.

Sad workaround: (un)pickle anything sent via this queue yourself.
History
Date User Action Args
2022-04-11 14:59:29adminsetgithub: 84376
2020-04-05 10:55:01Sander Landcreate