Currently applying functools.partial to a callable that is already functools.partial object results in a nested object:

>>> from functools import partial
>>> def f(a,b,c): pass
... 
>>> p = partial(partial(f, 1), 2)
>>> p.func, p.args
(<functools.partial object at 0x100431d60>, (2,))


Proposed patch makes partial(partial(f, 1), 2) return partial(f, 1, 2) instead:
>>> p.func, p.args
(<function f at 0x10055d3a8>, (1, 2))

This patch is partially (no pun intended) motivated by a patch submitted by Christophe Simonis for issue4331. Christophe's patch flattens nested partials for a specific case of using partials as bound methods.

As proposed, the patch will enable flattening for subclasses of functools.partial, but will return a baseclass instance.  Flattening will also discard any state attached to the nested partial such as __name__, __doc__, etc or any subclass data.  I believe this is the right behavior, but this caveat is the reason I classify this patch as a "feature request" rather than "performance" or "resource usage".

Flattening should only happen for instances of the exact type. When people create subclasses, there's usually a reason for it.

Antoine> Flattening should only happen for instances of the exact type.

I am attaching a variant of the patch that will only flatten if both nested and wrapping partial is of the exact type. Other possibilities would include flattening partial_subtype(f, ...) if type(f) == partial and if type(f) == partial_subtype, but I'll present only the least and most conservative variants.

Antoine> When people create subclasses [of partial type], there's usually a reason for it.

It is hard not to agree with this thesis, but I don't see how it follows that subclass writers will not benefit from flattening their instances.  Can you suggest a use case where flattening in functools.partial subtype would be undesirable?

I am attaching a patch, issue7830.diff, that takes an ultra-concervative approach: partials are only flattened if both outer and inner are of exact functools.partial type and the inner partial does not have __dict__.

Applies cleanly on the py3k branch at r83069, the tests work correctly (fail before applying the patch, success afterwards), and, to the best of my C-API knowledge, the C code is alright.

Oh, and it behaves as described...

Python 3.2a0 (py3k:83069M, Jul 23 2010, 12:40:49) 
[GCC 4.2.1 (Apple Inc. build 5659)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(a, b, c): pass
... 
>>> from functools import partial
>>> p1 = partial(foo, 1)
>>> p1.func, p1.args
(<function foo at 0x1004531e8>, (1,))
>>> p2 = partial(foo, 2)
>>> p2.func, p2.args
(<function foo at 0x1004531e8>, (2,))

Oops, used it wrong (but it still works correctly).


>>> p2 = partial(p1, 2)
>>> p2.func, p2.args
(<function foo at 0x10051da68>, (1, 2))

Since I am the OP of this patch, I would like a +1 from another developer before checking this in.  (Or a couple of -1s before rejecting:-)


Antoine,

Does the latest patch address your concerns?

Virgil,

If you care enough about this feature, please post a summary on python-dev and ask for comments.

Antoine> Flattening should only happen for instances of the exact type.

FWIW, I agree with Antoine. You cannot know in advance whether a partial-subclass has semantics that need to be preserved when flattening.

> FWIW, I agree with Antoine. You cannot know in advance whether a
> partial-subclass has semantics that need to be preserved when
> flattening.


Raymond,

I have actually conceded this point to Antoine.  See msg108980 above.  Not only the latest patch preserves partial-subclasses, it also foregoes the optimization if there is anything in __dict__ of the inner partial.

It looks like we have a consensus on the features, the remaining question is whether this is enough of an optimization to justify adding extra code.

Alexander, I don't see anything wrong with patch, nor anything compelling about it either.   It's your choice whether or not to apply.

The original motivation for the patch was that if partial() objects are guaranteed to be flat, it would simplify code that deals with them.  See issue4331 for one example.

With a "conservative" patch, however, it will still be possible to create nested partials and the code consuming partials should be ready for that.