Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

incompatible: unittest.mock.sentinel and multiprocessing.Pool.map() #73415

Closed
thatneat mannequin opened this issue Jan 10, 2017 · 4 comments
Closed

incompatible: unittest.mock.sentinel and multiprocessing.Pool.map() #73415

thatneat mannequin opened this issue Jan 10, 2017 · 4 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@thatneat
Copy link
Mannequin

thatneat mannequin commented Jan 10, 2017

BPO 29229
Nosy @applio, @thatneat
Superseder
  • bpo-20804: Sentinels identity lost when pickled (unittest.mock)
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-01-11.00:56:45.889>
created_at = <Date 2017-01-10.19:22:09.047>
labels = ['type-bug', 'library']
title = 'incompatible: unittest.mock.sentinel and multiprocessing.Pool.map()'
updated_at = <Date 2017-01-11.00:56:45.876>
user = 'https://github.com/thatneat'

    bugs.python.org fields:

    activity = <Date 2017-01-11.00:56:45.876>
actor = 'jason.curtis'
assignee = 'none'
closed = True
closed_date = <Date 2017-01-11.00:56:45.889>
closer = 'jason.curtis'
components = ['Library (Lib)']
creation = <Date 2017-01-10.19:22:09.047>
creator = 'jason.curtis'
dependencies = []
files = []
hgrepos = []
issue_num = 29229
keywords = []
message_count = 4.0
messages = ['285146', '285149', '285151', '285174']
nosy_count = 2.0
nosy_names = ['davin', 'jason.curtis']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '20804'
type = 'behavior'
url = 'https://bugs.python.org/issue29229'
versions = ['Python 3.4']
    @thatneat
    Copy link
    Mannequin Author

    thatneat mannequin commented Jan 10, 2017

    When a sentinel object from unittest.mock.sentinel is passed through a multiprocessing.Pool.map, I expect it to still be comparable.

    As a user, I'd like to be able to write a unit test using sentinels on my unparallelized code, and then see that the test still passes after I parallelize the code using multiprocessing, so that I can make use of sentinels in regression testing.

    Example:

    from unittest import mock
import multiprocessing


def identity(x):
    return x

with multiprocessing.Pool() as pool:
    multiprocessed = list(pool.map(identity, [mock.sentinel.foo]))

assert identity(mock.sentinel.foo) == mock.sentinel.foo  # Passes
assert multiprocessed[0] == mock.sentinel.foo  # Fails

    @thatneat thatneat mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jan 10, 2017
    @applio
    Copy link
    Member

    applio commented Jan 10, 2017

    This arises from the behavior of pickle (which is used by default in multiprocessing to serialize objects sent to / received from other processes in data exchanges), as seen with Python 3.6:

    >>> import pickle
>>> x = pickle.dumps(mock.sentinel.foo)
>>> x
b'\x80\x03cunittest.mock\n_SentinelObject\nq\x00)\x81q\x01}q\x02X\x04\x00\x00\x00nameq\x03X\x03\x00\x00\x00fooq\x04sb.'
>>> pickle.loads(x)
sentinel.foo
>>> pickle.loads(x) == mock.sentinel.foo
False

    @applio
    Copy link
    Member

    applio commented Jan 10, 2017

    I think this should be regarded as a duplicate of bpo-20804 though discussion in bpo-14577 is also related/relevant.

    @thatneat
    Copy link
    Mannequin Author

    thatneat mannequin commented Jan 11, 2017

    sounds right; closing as a duplicate of bpo-20804

    @thatneat thatneat mannequin closed this as completed Jan 11, 2017
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @applio