classification
Title: incompatible: unittest.mock.sentinel and multiprocessing.Pool.map()
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Sentinels identity lost when pickled (unittest.mock)
View: 20804
Assigned To: Nosy List: davin, jason.curtis
Priority: normal Keywords:

Created on 2017-01-10 19:22 by jason.curtis, last changed 2017-01-11 00:56 by jason.curtis. This issue is now closed.

Messages (4)
msg285146 - (view) Author: Jason Curtis (jason.curtis) * Date: 2017-01-10 19:22
When a sentinel object from unittest.mock.sentinel is passed through a multiprocessing.Pool.map, I expect it to still be comparable. 

As a user, I'd like to be able to write a unit test using sentinels on my unparallelized code, and then see that the test still passes after I parallelize the code using multiprocessing, so that I can make use of sentinels in regression testing.


Example:
```
from unittest import mock
import multiprocessing


def identity(x):
    return x

with multiprocessing.Pool() as pool:
    multiprocessed = list(pool.map(identity, [mock.sentinel.foo]))

assert identity(mock.sentinel.foo) == mock.sentinel.foo  # Passes
assert multiprocessed[0] == mock.sentinel.foo  # Fails
```
msg285149 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2017-01-10 19:41
This arises from the behavior of pickle (which is used by default in multiprocessing to serialize objects sent to / received from other processes in data exchanges), as seen with Python 3.6:

>>> import pickle
>>> x = pickle.dumps(mock.sentinel.foo)
>>> x
b'\x80\x03cunittest.mock\n_SentinelObject\nq\x00)\x81q\x01}q\x02X\x04\x00\x00\x00nameq\x03X\x03\x00\x00\x00fooq\x04sb.'
>>> pickle.loads(x)
sentinel.foo
>>> pickle.loads(x) == mock.sentinel.foo
False
msg285151 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2017-01-10 19:49
I think this should be regarded as a duplicate of issue20804 though discussion in issue14577 is also related/relevant.
msg285174 - (view) Author: Jason Curtis (jason.curtis) * Date: 2017-01-11 00:56
sounds right; closing as a duplicate of issue20804
History
Date User Action Args
2017-01-11 00:56:45jason.curtissetstatus: open -> closed
resolution: duplicate
messages: + msg285174
2017-01-10 19:49:59davinsetsuperseder: Sentinels identity lost when pickled (unittest.mock)
messages: + msg285151
2017-01-10 19:41:17davinsetnosy: + davin
messages: + msg285149
2017-01-10 19:22:09jason.curtiscreate