classification
Title: multiprocessing.JoinableQueue requires new kwarg
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.4
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Cryptic error when subclassing multiprocessing classes
View: 19895
Assigned To: Nosy List: dan.oreilly, larkost, lee.clemens, sbt
Priority: normal Keywords:

Created on 2014-04-27 17:12 by lee.clemens, last changed 2016-05-01 13:29 by berker.peksag. This issue is now closed.

Messages (4)
msg217289 - (view) Author: Lee Clemens (lee.clemens) Date: 2014-04-27 17:12
Not mentioned (at least not specifically) in the release notes, multiprocessing.JoinableQueue now requires 'ctx' keyword argument:
  def __init__(self, maxsize=0, *, ctx):

This causes an application calling JoinableQueue() to work with 3.3.2 (my single test) to work, but not with 3.4.0

TypeError: __init__() missing 1 required keyword-only argument: 'ctx'

The documentation is also incorrect: https://docs.python.org/3.4/library/multiprocessing.html#multiprocessing.JoinableQueue
msg217290 - (view) Author: Lee Clemens (lee.clemens) Date: 2014-04-27 17:25
Same issue (ctx keyword) occurs with multiprocessing.queues.SimpleQueue
msg223869 - (view) Author: Dan O'Reilly (dan.oreilly) * Date: 2014-07-24 18:49
How are you importing JoinableQueue? You'll see this error if you import it from multiprocessing.queues instead of directly from multiprocessing. This is because multiprocessing.JoinableQueue is now a function:


    def JoinableQueue(self, maxsize=0):
        '''Returns a queue object'''
        from .queues import JoinableQueue
        return JoinableQueue(maxsize, ctx=self.get_context())

It provides the required context argument for you. Make sure your application is doing "from multiprocessing import JoinableQueue", rather than "from multiprocessing.queues import JoinableQueue".
msg230716 - (view) Author: larkost (larkost) Date: 2014-11-05 22:07
We just got bitten by this issue because we are trying to be compatible across 2.x and 3.x (including 3.0-3.2). For anyone who runs into the "missing 1 required keyword-only argument: 'ctx'" here is an import statement that works:

try:
    from multiprocessing import SimpleQueue
except ImportError:
    from multiprocessing.queues import SimpleQueue

Replace SimpleQueue with JoinableQueue if you need that. Importing in the other order will wind you up in problems in 3.4.2+.
History
Date User Action Args
2019-04-26 19:39:46SilentGhostlinkissue19895 superseder
2016-05-01 13:29:11berker.peksagsetstatus: open -> closed
superseder: Cryptic error when subclassing multiprocessing classes
resolution: duplicate
stage: resolved
2014-11-05 22:07:09larkostsetnosy: + larkost
messages: + msg230716
2014-07-24 18:50:35dan.oreillysettype: compile error -> behavior
components: + Library (Lib), - Interpreter Core
2014-07-24 18:49:24dan.oreillysetnosy: + dan.oreilly
messages: + msg223869
2014-04-27 17:28:15Claudiu.Popasetnosy: + sbt
2014-04-27 17:25:45lee.clemenssetmessages: + msg217290
2014-04-27 17:12:14lee.clemenscreate