classification
Title: multiprocessing AuthenticationError "digest sent was rejected" when pickling proxy
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.7, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jnoller, nirai, peterhunt, xhantu
Priority: normal Keywords:

Created on 2009-12-14 07:53 by peterhunt, last changed 2011-11-15 08:27 by xhantu.

Files
File name Uploaded Description Edit
multiprocessing_bug.py peterhunt, 2009-12-14 07:53
Messages (4)
msg96371 - (view) Author: Pete Hunt (peterhunt) Date: 2009-12-14 07:53
When pickling a proxy object (such as a list) created on a client, an 
exception is thrown. Example attached.

Python 2.6.4 Mac OS X 10.6

p
msg96372 - (view) Author: Pete Hunt (peterhunt) Date: 2009-12-14 08:04
UPDATE: this example WORKS if you remove "authkey" - so it seems to be a 
problem with authentication.
msg147657 - (view) Author: (xhantu) Date: 2011-11-15 08:26
Confirmed for Python 2.7.1 on Ubuntu.

Problematic are the __reduce__ methods of multiprocessing.process.AuthenticationString and multiprocessing.managers.BaseProxy. Pickling of an authkey in BaseProxy is only done and allowed when Popen.thread_is_spawning() is True. The comments state this is done because of security reasons.

If you pass proxies with multiprocessing after process initialization then no authkey is available after unpickle, a random authkey will be used with results in an digest error. Because of this fallback in case no authkey is available, using None as authkey didn't work in my case.

This bug looks like a design flaw of multiprocessing. If security was really a concern, encryption should be used along with authentication for communication. And passing None as authkey should disable authentication as stated in the documentation and should not use a random authkey.

Disabling the Popen.thread_is_spawning() checks allows passing of proxies. I have done this during runtime by replacing the __reduce__ methods of the affected classes to avoid patching the installed python.
msg147658 - (view) Author: (xhantu) Date: 2011-11-15 08:27
forgot to set version in classification
History
Date User Action Args
2011-11-15 08:27:40xhantusetmessages: + msg147658
versions: + Python 2.7
2011-11-15 08:26:08xhantusetnosy: + xhantu
messages: + msg147657
2009-12-25 07:46:05niraisetnosy: + nirai
2009-12-14 12:42:14r.david.murraysetpriority: normal
type: crash -> behavior
2009-12-14 12:41:22r.david.murraysetnosy: + jnoller
2009-12-14 08:04:07peterhuntsettype: crash
messages: + msg96372
2009-12-14 07:53:53peterhuntcreate