classification
Title: incomplete signature with help function using typing
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: David E. Franco G., Spencer Brown, gvanrossum, levkivskyi, python-dev, yselivanov
Priority: normal Keywords: patch

Created on 2016-09-07 02:19 by David E. Franco G., last changed 2016-10-22 15:01 by gvanrossum. This issue is now closed.

Files
File name Uploaded Description Edit
typing-pydoc.diff levkivskyi, 2016-10-22 14:05 review
Messages (15)
msg274702 - (view) Author: David E. Franco G. (David E. Franco G.) Date: 2016-09-07 02:19
the issue is that when calling help on a function annotated with typing, all the relevant information is lost, for example

from typing import List, Any, Iterator, Tuple

def foo(data:List[Any]) -> Iterator[ Tuple[int,Any] ]:
    ...
 
when calling help on that

>>> help(foo)
Help on function foo in module __main__:

foo(data:typing.List) -> typing.Iterator

>>> 

all the information is lost, the output should look like this

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[ Tuple[int, Any] ]:

>>> 

where all the information that I put in the annotation is preserved and the typing.* are eliminated since they only add unnecessary noise

while reporting this issue in the typing module (https://github.com/python/typing/issues/279) I was told that is a bug with the inspect module and that help may need modification.

Thank for your time.
msg274734 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-07 03:13
It seems the output produced here is generated by inspect.signature(), which is called by pydoc in this case (in both versions of docroutine()).

I don't know if the right thing to do is to change inspect.signature() here, or to change pydoc to use something else to format the argument list.
msg274787 - (view) Author: Spencer Brown (Spencer Brown) * Date: 2016-09-07 09:59
More precisely, the issue is with inspect.formatannotation(), which overrides/ignores the repr if the annotation is an instance of type. Perhaps that should be changed to also check that __repr__ is type's repr.
msg274814 - (view) Author: David E. Franco G. (David E. Franco G.) Date: 2016-09-07 14:18
as that is the case, how about this as a solution:

def formatannotation(annotation, base_module=None):
    if isinstance(annotation, type):
        if annotation.__module__ in ('builtins', base_module):
            return annotation.__qualname__
        elif annotation.__module__ in ('typing', base_module):       
            return repr(annotation).replace("typing.","")
        return annotation.__module__+'.'+annotation.__qualname__
    return repr(annotation)

the same way that it check for builtins, do it for typing and clean up a little. 

With that change the result with the example is

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[Tuple[int, Any]]

>>>
msg274879 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-07 20:18
That sounds a fine solution (except the elif should just test for `in
'typing'`). Can one of you prepare a patch? I think it should be fine
to fix this in 3.5 as well.

There should be a unit test for this.
msg274881 - (view) Author: Spencer Brown (Spencer Brown) * Date: 2016-09-07 20:42
It might be better to just change the if statement to 'if isinstance(annotation, type) and type(annotation).__repr__ is type.__repr__:'. That would make it fallback for any metaclass which overrides repr, instead of special-casing typing. That also ensures 'typing.' is still in the name, since these aren't builtins.
msg274889 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-07 20:54
I've lost you -- why don't you upload a patch?
msg274904 - (view) Author: David E. Franco G. (David E. Franco G.) Date: 2016-09-07 22:48
I think that removing the "typing." is for the best, with the example above 

>>> help(foo)
Help on function foo in module __main__:

foo(data:typing.List[typing.Any]) -> typing.Iterator[typing.Tuple[int, typing.Any]]

>>> 


leaving the "typing." produce result that I think are ugly and distracting, not only that, is unnecessary long to convey the same information that can be in a more neat way without it, and more so while more complicated/long the signature is. 

just compare the above with this

>>> help(foo)
Help on function foo in module __main__:

foo(data:List[Any]) -> Iterator[Tuple[int, Any]]:

>>> 

which is a clear winner to me.

Or perhaps alongside modifying inspect.formatannotation also change the __repr__ in typing to exclude the `typing.` or like with for instance TypeVar produce a repr that include some marker instead, like ~ and in that way indicate that one is using typing object resulting in something like this

>>> help(foo)
Help on function foo in module __main__:

foo(data:~List[Any]) -> ~Iterator[~Tuple[int, ~Any]]:

>>> 

which is a little weird but still neat
msg279190 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2016-10-22 14:05
Here is the patch according to the discussion (modifying inspect).

I didn't change the rendering of docs for classes (neither stripped 'typing.' nor changed __bases__ to __orig_bases__). First, collections.abc.X are widely used as base classes, so that plain Mapping could be confused with collections.abc.Mapping. Second, seeing the actual runtime type-erased bases suggests that one should use isinstance() and issubclass() with those (not with, e.g.,  Mapping[int, str], the latter will raise TypeError).
msg279191 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-10-22 14:35
Hm, I actually like the original proposal better. Perhaps collections.abc.Mapping is more common than typing.Mapping, but is it more common *in function annotations*? I don't think so.

Also, I like showing e.g. Iterator[Tuple[int, Any]] rather than just Iterator. This is documentation we're talking about, and the parameter types are very useful as documentation. (However, abbreviating List[Any] as List is fine, since they mean the same thing.)
msg279192 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2016-10-22 14:41
For function annotations I did as originally proposed. In my previous comment I was talking about documentation for classes. For example:

class C(Generic[T], Mapping[int, str]): ...
pydoc.render_doc(C)

will show "class C(typing.Mapping)".

while for function annotations typing is indeed much more common so that pydoc.render_doc(foo) will show

foo(data: List[Any]) -> Iterator[Tuple[int, Any]]
msg279193 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-10-22 14:53
OK, sounds good then. I guess most of the work was in typing.py, not in inspect. :-)
msg279194 - (view) Author: Ivan Levkivskyi (levkivskyi) * (Python committer) Date: 2016-10-22 14:55
Actually, for classes, it is probably worth adding a separate section "Generic type info" that will render information using __orig_bases__, __parameters__, and __args__. At the same time the "header" will be the same as now, listing runtime __bases__.

What do you think about this? Should I open a separate issue?
msg279195 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-10-22 14:58
New changeset dc030d15f80d by Guido van Rossum in branch '3.5':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L.
https://hg.python.org/cpython/rev/dc030d15f80d

New changeset 3937502c149d by Guido van Rossum in branch '3.6':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L. (3.5->3.6)
https://hg.python.org/cpython/rev/3937502c149d

New changeset 62127e60e7b0 by Guido van Rossum in branch 'default':
Issue #27989: Tweak inspect.formatannotation() to improve pydoc rendering of function annotations. Ivan L. (3.6->3.7)
https://hg.python.org/cpython/rev/62127e60e7b0
msg279197 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-10-22 15:01
Honestly I think pydoc is already too verbose. It would be better if the class header looked more like what was written in the source code -- that is the most compact way to render it. I say open a separate issue since this issue is about functions.
History
Date User Action Args
2016-10-22 15:01:01gvanrossumsetmessages: + msg279197
2016-10-22 14:58:34gvanrossumsetstatus: open -> closed
resolution: fixed
stage: resolved
2016-10-22 14:58:11python-devsetnosy: + python-dev
messages: + msg279195
2016-10-22 14:55:22levkivskyisetmessages: + msg279194
2016-10-22 14:53:15gvanrossumsetmessages: + msg279193
2016-10-22 14:41:33levkivskyisetmessages: + msg279192
2016-10-22 14:35:18gvanrossumsetmessages: + msg279191
2016-10-22 14:05:28levkivskyisetfiles: + typing-pydoc.diff

nosy: + yselivanov
messages: + msg279190

keywords: + patch
2016-09-07 22:48:27David E. Franco G.setmessages: + msg274904
2016-09-07 22:16:02levkivskyisetnosy: + levkivskyi
2016-09-07 20:54:31gvanrossumsetmessages: + msg274889
2016-09-07 20:42:23Spencer Brownsetmessages: + msg274881
2016-09-07 20:18:43gvanrossumsetmessages: + msg274879
2016-09-07 14:18:47David E. Franco G.setmessages: + msg274814
2016-09-07 09:59:07Spencer Brownsetnosy: + Spencer Brown
messages: + msg274787
2016-09-07 03:13:43gvanrossumsetnosy: + gvanrossum
messages: + msg274734
2016-09-07 02:19:50David E. Franco G.create