classification
Title: functools.partialmethod should look more like what it's impersonating.
Type: enhancement Stage:
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Aaron Hall, r.david.murray, skip.montanaro, smarie, xtreak
Priority: normal Keywords: patch

Created on 2017-04-21 14:27 by skip.montanaro, last changed 2019-09-20 15:13 by smarie.

Files
File name Uploaded Description Edit
partial3.py skip.montanaro, 2017-04-21 23:43
ft.diff skip.montanaro, 2017-04-21 23:43
Messages (6)
msg292050 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-04-21 14:27
I needed to create a partial method in Python 2.7, so I grabbed functools.partialmethod from a Python 3.5.2 install. For various reasons, one of the reasons I wanted this was to suck in some methods from a delegated class so they appeared in dir() and help() output on the primary class (the one containing the partialmethod objects). Suppose I have

class Something:
  def meth(self, arg1, arg2):
    "meth doc"
    return arg1 + arg2

then in the metaclass for another class I construct an attribute (call it "mymeth") which is a partialmethod object. When I (for example), run pydoc, that other class's attribute appears as:

    mymeth = <functools.partial object>

It would be nice if it at least included the doc string from meth, something like:

    mymeth = <functools.partial object>
        meth doc

Even better would be a proper signature:

    mymeth(self, arg1, arg2)
        meth doc

In my copy of functools.partialmethod, I inserted an extra line in __get__, right after the call to partial():

    results.__doc__ = self.func.__doc__

That helps a bit, as I can

    print("mymeth doc:", inst.mymeth.__doc__)

and see

    mymeth doc: meth doc

displayed. That's not enough for help()/pydoc though.

I suspect the heavy lifting will have to be done in pydoc.Doc.document(). inspect.isroutine() returns False for functools.partial objects. I also see _signature_get_partial() in inspect.py. That might be the source of the problem. When I create a partialmethod object in my little example, it actually looks like a functools.partial object, not a partialmethod object. It's not clear that this test:

    if isinstance(partialmethod, functools.partialmethod):

in inspect._signature_from_callable() is testing for the correct type.

Apologies that I can't easily provide a detailed example. My Python 2.x metaclass example (where I'm smashing methods from one class into another) doesn't work in Python 3.x for some reason, the whole partialmethod thing isn't available in Python 2.x (inspect doesn't know about partialmethod or partial) and it's not really a "hello world"-sized example anyway. I'll beat on things a bit more to try and craft a workable Python 3.x example.
msg292051 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-04-21 14:39
Yes, please do provide an example. Your final words do not make a convincing case that this is a problem in python3 :)
msg292070 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-04-21 18:04
Yeah, sorry about that. I work in an environment where I can't "eject" any
code from my work computer. I've come up with a simple Python3 example, but
it will have to wait until I can recreate it from scratch on my home
computer.
msg292088 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-04-21 23:49
Again, my apologies for the crappy initial bug report. Hopefully this comment and the two files I just attached demonstrate what I am getting at.

I just uploaded a stupid little example, partial3.py. Stupid, but still, it demonstrates part of how I think docstrings on these partial methods could be improved. If you run it (I'm using Python 3.6.1), note that the doc strings for the sum method (instance and class), look wrong:

Child.sum doc: None
c.sum doc: partial(func, *args, **keywords) - new function with partial application
    of the given arguments and keywords.

The file, ft.diff, includes a one-line patch to partialmethod.__get__ which corrects the docstring for the instance of the Child class:

Child.sum doc: None
c.sum doc: sum doc

I haven't looked to see where the docstring of Child.sum could be set, but I believe it should be fairly straightforward for someone more familiar with this code. Also, the patch doesn't improve the output of pydoc:

partial3.Child = class Child(builtins.object)
 |  Methods defined here:
 |  
 |  diff(self, arg1, arg2)
 |      diff doc
 |  
 |  sum = _method(self, arg2)
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
msg329348 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-11-06 07:40
I did some analysis with the given example script. Related issue where Aaron has some analysis : issue12154 . The patch doesn't break anything since there are no tests for this that also could be added.

1. When we do inspect.getdoc(c.sum) it looks for obj.__doc__ (c.sum.__doc__) returning partialmethod.__doc__ ("new function with partial...") instead of checking if c.sum is a partial method and  returning obj.func.__doc__ which contains the relevant doc ("sum doc"). Thus getdoc needs to check before trying for obj.__doc__ if the obj is a partialmethod or partial object thus returning the original function object.

2. When we do inspect.getdoc(Child.sum) it looks for obj.__doc__ (Child.sum.__doc__) and since Child.sum is a partialmethod which has __get__ overridden it calls _make_unbound_method that returns a _method object with no doc and thus returning None. partialmethod object copies objects from the given function at https://github.com/python/cpython/blob/f1b9ad3d38c11676b45edcbf2369239bae436e56/Lib/functools.py#L368 and the actual object is returned at https://github.com/python/cpython/blob/f1b9ad3d38c11676b45edcbf2369239bae436e56/Lib/functools.py#L401 . Here self.func has the original function in this case Base.sum and _method._partialmethod has reference to Base.sum which contains the relevant docs but _method itself has no docs thus pydoc doesn't get any docs. So we can set _method.__doc__ = self.func.__doc__ and getdoc can pick up the docs.

sample script with

print(pydoc.render_doc(Child))
print(pydoc.render_doc(c))
print(inspect.getdoc(c.sum)) # Need to patch getdoc before it checks for obj.__doc__ to check if the c.sum is an instance of partialmethod or partial
print(inspect.getdoc(Child.sum)) # Need to patch functools.partialmethod._make_unbound_method to copy the docs to _method

Output :

```
Python Library Documentation: class Child in module __main__

class Child(builtins.object)
 |  Methods defined here:
 |
 |  diff(self, arg1, arg2)
 |      diff doc
 |
 |  sum = _method(self, arg2)
 |      sum doc
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables (if defined)
 |
 |  __weakref__
 |      list of weak references to the object (if defined)

Python Library Documentation: Child in module __main__ object

class Child(builtins.object)
 |  Methods defined here:
 |
 |  diff(self, arg1, arg2)
 |      diff doc
 |
 |  sum = _method(self, arg2)
 |      sum doc
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables (if defined)
 |
 |  __weakref__
 |      list of weak references to the object (if defined)

sum doc
sum doc

```

patch : 

diff --git a/Lib/functools.py b/Lib/functools.py
index ab7d71e126..751f67fcd0 100644
--- a/Lib/functools.py
+++ b/Lib/functools.py
@@ -398,6 +398,7 @@ class partialmethod(object):
             return self.func(*call_args, **call_keywords)
         _method.__isabstractmethod__ = self.__isabstractmethod__
         _method._partialmethod = self
+        _method.__doc__ = self.func.__doc__ or self.__doc__
         return _method

     def __get__(self, obj, cls):
diff --git a/Lib/inspect.py b/Lib/inspect.py
index b8a142232b..2c796546b2 100644
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -600,6 +600,9 @@ def getdoc(object):
     All tabs are expanded to spaces.  To clean up docstrings that are
     indented to line up with blocks of code, any whitespace than can be
     uniformly removed from the second line onwards is removed."""
+    if isinstance(object, (functools.partialmethod, functools.partial)):
+        return object.func.__doc__
+
     try:
         doc = object.__doc__
     except AttributeError:
msg352854 - (view) Author: Sylvain Marie (smarie) * Date: 2019-09-20 15:13
For future reference if this topic re-opens, there is now an alternative here:

https://smarie.github.io/python-makefun/#removing-parameters-easily

Note: it relies on a dynamic `compile` statement so of course it is less optimal than the one in functools. But at least it can serve as a reference...
History
Date User Action Args
2019-09-20 15:13:00smariesetnosy: + smarie
messages: + msg352854
2018-11-06 07:40:29xtreaksetnosy: + xtreak
messages: + msg329348
2018-05-29 01:06:25Aaron Hallsetnosy: + Aaron Hall
2017-04-21 23:49:55skip.montanarosetmessages: + msg292088
2017-04-21 23:43:54skip.montanarosetfiles: + ft.diff
keywords: + patch
2017-04-21 23:43:43skip.montanarosetfiles: + partial3.py
2017-04-21 18:04:30skip.montanarosetmessages: + msg292070
2017-04-21 14:39:37r.david.murraysetnosy: + r.david.murray
messages: + msg292051
2017-04-21 14:27:51skip.montanarocreate