classification
Title: concurrent.futures: Executor.submit keyword arguments may not be called 'fn' (or 'self')
Type: behavior Stage:
Components: Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: duplicate
Dependencies: Superseder: **kwargs unnecessarily restricted in concurrent.futures 'submit' API
View: 10918
Assigned To: bquinlan Nosy List: bquinlan, brian.curtin, mark.dickinson, pitrou, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2012-09-18 15:32 by mark.dickinson, last changed 2013-08-06 13:28 by mark.dickinson. This issue is now closed.

Files
File name Uploaded Description Edit
futures.patch mark.dickinson, 2012-09-18 15:59 review
Messages (8)
msg170650 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-09-18 15:32
The submit methods of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExectutor raise TypeError when submitting a callable with a keyword argument named 'fn' or 'self':

    Python 3.3.0rc2+ (default:3a880d640981, Sep 18 2012, 16:29:28) 
    [GCC 4.2.1 (Apple Inc. build 5664)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import concurrent.futures
    >>> def awkward(*, fn): return fn * 1729
    ... 
    >>> with concurrent.futures.ThreadPoolExecutor(1) as e:
    ...     e.submit(awkward, fn=3)
    ... 
    Traceback (most recent call last):
      File "<stdin>", line 2, in <module>
    TypeError: submit() got multiple values for argument 'fn'


An obvious solution is to change the declarations of the submit methods from:

    def submit(self, fn, *args, **kwargs):
        ...

to

    def submit(*args, **kwargs):
        self, fn, *args = args


I don't think this is quite good enough, since it'll introduce a regression for anyone who was doing executor.submit(fn=...).
msg170652 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-09-18 15:59
Here's a patch.  The solution is ugly enough that I'm wondering whether this is even worth fixing.
msg170686 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-09-18 22:35
The patch looks ok to me. At least passing "fn" as keyword arg should be fixed. Passing "self" as keyword arg admittedly sounds a bit awkward.
msg170721 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-09-19 10:40
Ah, I added the wrong Brian to the nosy.  Sorry, Brian C.
msg173436 - (view) Author: Brian Quinlan (bquinlan) * (Python committer) Date: 2012-10-21 09:13
This has come up before. Did you actually bang into this? Because the fix seems pretty ugly to me and the work-around (using functools.partial) is pretty easy.

But, if people are actually hitting this, then your is probably the best that we can do.
msg173546 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-10-22 17:35
> Did you actually bang into this?

No directly, no:  I'd hit a similar problem in some of our own code, leading to a signature change from "def do_later(fn, *args, **kwargs)" to "def do_later(fn, arg, kwargs)".  Then it occurred to me to wonder how Executor.submit handled the issue.

The workaround seems fine;  feel free to close as "won't fix".
msg173562 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2012-10-22 20:09
There are many other cases of such issue: collections.namedtuple, contextlib.contextmanager, profile.Profile.runcall, etc.
msg194538 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-08-06 13:02
Closing this.
History
Date User Action Args
2013-08-06 13:28:47mark.dickinsonsetsuperseder: **kwargs unnecessarily restricted in concurrent.futures 'submit' API
resolution: wont fix -> duplicate
2013-08-06 13:02:47mark.dickinsonsetstatus: open -> closed
resolution: wont fix
messages: + msg194538
2012-10-22 20:09:28serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg173562
2012-10-22 17:35:11mark.dickinsonsetmessages: + msg173546
2012-10-21 09:13:40bquinlansetmessages: + msg173436
2012-09-19 10:40:45mark.dickinsonsetmessages: + msg170721
2012-09-18 22:35:17pitrousetnosy: + pitrou
messages: + msg170686
2012-09-18 22:16:25brian.curtinsetassignee: bquinlan

nosy: + bquinlan
2012-09-18 22:07:48mark.dickinsonsetnosy: + brian.curtin
2012-09-18 15:59:42mark.dickinsonsetfiles: + futures.patch
keywords: + patch
messages: + msg170652
2012-09-18 15:32:48mark.dickinsoncreate