Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiprocessing.JoinableQueue requires new kwarg #65566

Closed
leeclemens mannequin opened this issue Apr 27, 2014 · 4 comments
Closed

multiprocessing.JoinableQueue requires new kwarg #65566

leeclemens mannequin opened this issue Apr 27, 2014 · 4 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@leeclemens
Copy link
Mannequin

leeclemens mannequin commented Apr 27, 2014

BPO 21367
Superseder
  • bpo-19895: Cryptic error when subclassing multiprocessing classes
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2016-05-01.13:29:11.321>
    created_at = <Date 2014-04-27.17:12:14.091>
    labels = ['type-bug', 'library']
    title = 'multiprocessing.JoinableQueue requires new kwarg'
    updated_at = <Date 2016-05-01.13:29:11.320>
    user = 'https://bugs.python.org/leeclemens'

    bugs.python.org fields:

    activity = <Date 2016-05-01.13:29:11.320>
    actor = 'berker.peksag'
    assignee = 'none'
    closed = True
    closed_date = <Date 2016-05-01.13:29:11.321>
    closer = 'berker.peksag'
    components = ['Library (Lib)']
    creation = <Date 2014-04-27.17:12:14.091>
    creator = 'lee.clemens'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 21367
    keywords = []
    message_count = 4.0
    messages = ['217289', '217290', '223869', '230716']
    nosy_count = 4.0
    nosy_names = ['sbt', 'lee.clemens', 'larkost', 'dan.oreilly']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '19895'
    type = 'behavior'
    url = 'https://bugs.python.org/issue21367'
    versions = ['Python 3.4']

    @leeclemens
    Copy link
    Mannequin Author

    leeclemens mannequin commented Apr 27, 2014

    Not mentioned (at least not specifically) in the release notes, multiprocessing.JoinableQueue now requires 'ctx' keyword argument:
    def __init__(self, maxsize=0, *, ctx):

    This causes an application calling JoinableQueue() to work with 3.3.2 (my single test) to work, but not with 3.4.0

    TypeError: __init__() missing 1 required keyword-only argument: 'ctx'

    The documentation is also incorrect: https://docs.python.org/3.4/library/multiprocessing.html#multiprocessing.JoinableQueue

    @leeclemens leeclemens mannequin added build The build process and cross-build interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Apr 27, 2014
    @leeclemens
    Copy link
    Mannequin Author

    leeclemens mannequin commented Apr 27, 2014

    Same issue (ctx keyword) occurs with multiprocessing.queues.SimpleQueue

    @danoreilly
    Copy link
    Mannequin

    danoreilly mannequin commented Jul 24, 2014

    How are you importing JoinableQueue? You'll see this error if you import it from multiprocessing.queues instead of directly from multiprocessing. This is because multiprocessing.JoinableQueue is now a function:

        def JoinableQueue(self, maxsize=0):
            '''Returns a queue object'''
            from .queues import JoinableQueue
            return JoinableQueue(maxsize, ctx=self.get_context())

    It provides the required context argument for you. Make sure your application is doing "from multiprocessing import JoinableQueue", rather than "from multiprocessing.queues import JoinableQueue".

    @danoreilly danoreilly mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error and removed interpreter-core (Objects, Python, Grammar, and Parser dirs) build The build process and cross-build labels Jul 24, 2014
    @larkost
    Copy link
    Mannequin

    larkost mannequin commented Nov 5, 2014

    We just got bitten by this issue because we are trying to be compatible across 2.x and 3.x (including 3.0-3.2). For anyone who runs into the "missing 1 required keyword-only argument: 'ctx'" here is an import statement that works:

    try:
    from multiprocessing import SimpleQueue
    except ImportError:
    from multiprocessing.queues import SimpleQueue

    Replace SimpleQueue with JoinableQueue if you need that. Importing in the other order will wind you up in problems in 3.4.2+.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant