Title: Idle: Use inspect.signature for calltips
Type: behavior Stage: resolved
Components: IDLE Versions: Python 3.7, Python 3.6
Status: closed Resolution: fixed
Dependencies: 20122 20223 20401 Superseder:
Assigned To: terry.reedy Nosy List: louielu, serhiy.storchaka, terry.reedy, veky
Priority: normal Keywords:

Created on 2013-12-06 00:21 by terry.reedy, last changed 2017-08-10 06:15 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1382 closed louielu, 2017-05-02 10:28
PR 2822 open louielu, 2017-07-23 08:53
PR 3053 merged terry.reedy, 2017-08-10 01:18
Messages (37)
msg205339 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-12-06 00:21
Change idlelib.CallTips.get_argspec to use inspect.signature, new in 3.3, instead of inspect.getfullargspec and inspect.formatargspec. One thing it handles better is a namedtuple class, which has a C-coded __init__ inherited from object a Python-coded __new__ written by namedtuple. Signature() will also handle C-coded functions if and when Argument Clinic (new in 3.4) adds signature information.

from collections import namedtuple
from inspect import signature

Point = namedtuple('Point', 'x y')
# '(_cls, x, y)'
# '(x, y)'

Note that str (called by print) is needed to get the desired string form.

There are tests in, but they should be converted to unittest, moved to idle_test/, changed to match new output detail, and expanded to include new examples.
msg208757 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-01-22 08:32
*17481 is about changing getargspec to use signature, at least as a backup. But that would not fix the example here. The calltips have been moved in 20122.
msg213878 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-03-17 17:17
I have not decided yet whether to apply to 3.4.
msg219203 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-05-27 05:53
My understanding is that when a builtin is converted, a) the docstring signature disappears, and b) getfullargspec calls str(signature). I need to check what now happens with .getfullargspec version .signature for uncoverted builtins and if any changes should be made now.
msg235154 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-02-01 09:30
.signature also handles wrapped functions better.

>>> def deco(f):
        import functools
        def ret(*args, **kw):
                return f(*args, **kw)
        return ret

>>> @deco
def f(n, k):
        return n + k*2

>>> import inspect as ip
>>> isf=ip.signature(f)
>>> print(isf)
(n, k)
>>> ip.formatargspec(*ip.getfullargspec(f))
'(*args, **kw)'
msg235156 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-02-01 09:50
The above suggests that the change might be applied to 3.4.  The difference surprises me because #17481 patched getfullargspec() to 'use' signature() 'behind the scenes', so I expected the result to be the same.  It seems that 'use' does not mean what I thought.  My hesitation is that only some builtins have been converted to use Argument Clinic, and Idle currently treats builtins special to get the signature from the docstring.  Treating just some builting special will be slightly awkward.
msg235157 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-02-01 10:12
An example should make my concern clearer. str(signature(int)) is '()' because int (and all other builtins) has not been converted whereas the Idle calltip is
  int(x=0) -> integer
  int(x, base=10) -> integer
I do not want to simply add a line with '()' to the top of the calltip, so a little thought is needed to improve the calltip for the rare cases above without degrading them for existing, more common cases.
msg288586 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-02-25 22:55
Several builtin functions have recently gotten the Arg clinic treatment.  So I think it is now time to switch.  Before closing #29653 as a duplicate, I discovered that inspect.signature also handles functools.partials correctly, while the other inspect functions do not.
msg292636 - (view) Author: Louie Lu (louielu) * Date: 2017-04-30 17:06
I'm now testing to change getfullargspect to signature. It came out that signature can't accept bound method with only _VAR_KEYWORD. This test case will gave a ValueError:

    >>> class C:
        def m2(**kw):
    >>> c = C()
    >>> ip.signature(c.m2)
   Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/", line 3002, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
  File "/usr/lib/python3.6/", line 2752, in from_callable
  File "/usr/lib/python3.6/", line 2169, in _signature_from_callable
    return _signature_bound_method(sig)
  File "/usr/lib/python3.6/", line 1759, in _signature_bound_method
    raise ValueError('invalid method signature')
ValueError: invalid method signature
msg292712 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-02 02:56
Thanks for the report. As Yuri pointed out in msg292701, and I verified, c.m2() raises "TypeError: meth() takes 0 positional arguments but 1 was given".

The purpose of get_argspec is to tell the user how to call the function without getting such a TypeError.  But this is not possible, at least in this case.  When signature raises "ValueError: invalid method signature", get_argspec should catch ValueError and put, for instance, "Function has an invalid method signature" in the box, instead of a signature that does not work, so users will know that the function cannot be called properly.  (Should message be red?)

An initial patch could just comment out this test.  I would like to see whether everything else passes.  Or have you, Louie, already done that?  
Signature can raise "TypeError if that type of object is not supported".  Does .signature return for all the other test cases?  I expect it should  if only called after current guard.

        ob_call = ob.__call__
    except BaseException:
        return argspec

Before committing a patch using .signature, the known possible exception (ValueError) must be caught.  This test should then be changed to verify the new behavior.

Possible future enhancement.  Current nothing happens after code like "1(" or "''(".  Should a box pop up saying (in red?) something like "Non-callable object before ("?
msg292719 - (view) Author: Louie Lu (louielu) * Date: 2017-05-02 06:47
Terry, only this case was failed, others in unittest work very well.
msg292721 - (view) Author: Louie Lu (louielu) * Date: 2017-05-02 07:00
ah, there are still some corner case about _first_param.sub(), some of them in signature will be like `(ci)`, not `(self, ci)`, and the regex will replace the first params to `()`, which is not correct.
msg292741 - (view) Author: Louie Lu (louielu) * Date: 2017-05-02 10:31
@Terry, the PR is created, it is now serving with signature. The two exception (object not callable, invalid method signature) will return the corresponding message.

The only thing I didn't figure out it how to change the color, so I didn't test for this feature.
msg292787 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-02 17:05
I will try to do that when I can.
msg292864 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-03 09:15
Later today I should be able to pull the PR to my machine to review and test, and recheck coverage change.
msg292964 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 05:59
I pulled your pr, did some minor edits, committed, and tried to push back following this line from the devguide Bootcamp page.

git push<contributor>/cpython <pr_XXX>:<branch_name>

F:\dev\cpython>git commit -m "Fix and stop repeating invalid signature message"
[pr_1382 bb46c06645] Fix and stop repeating invalid signature message
 2 files changed, 6 insertions(+), 5 deletions(-)

F:\dev\cpython>git push pr_1382:pr_1382
The authenticity of host ' (' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no': yes
Warning: Permanently added ',' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Did I do something wrong or do you need to allow me to push?

There current PR is incomplete.  The main reason to switch to signature() is to get the signature of C-coded functions that have been converted to use Argument Clinic.  But...

>>> import re
>>> import inspect as i
>>> str(i.signature(i.signature))
'(obj, *, follow_wrapped=True)'
>>> str(i.signature(re.sub))
'(pattern, repl, string, count=0, flags=0)'
>>> p = re.compile('')
>>> str(i.signature(p.sub))
'(repl, string, count=0)'
>>> p.sub(

After "i.signature(" and "re.sub(" (both 'function's), the popup has the signature.  After p.sub (a 'builtin_function_or_method'), there is only the beginning of the doc string, as before.  Signature is not called because of the current guard.  It must be removed.

Returning _invalid_signature (defined in my patch) on ValueError should only be done when the message is 'invalid method signature'. (Can there be an invalid function signature that compiles?  In not, removing 'method' (in my patch) is a mistake).  signature(int) raises "no signature found for builtin type <class 'int'>".  In such cases, argspec should be left as '', to be augmented by the docstring as now.

Tests need to include some builtins that have been converted.  Serhiy, can you suggest a suitable module with converted functions and class methods?  Any of the builtins?
msg292966 - (view) Author: Louie Lu (louielu) * Date: 2017-05-04 06:21
> Did I do something wrong or do you need to allow me to push?

Yes, I've added you to the collaborators on my fork (lulouie/cpython).

But in fact, new GitHub workflow will need to do something different, not just edit-and-commit. You will need to go to PR 1382 ( and start a new review. Click "file changes", when your cursor hovers on the source code line, it will pop out a blue plus icon on the line number, then click it to add a new comment on it, after that, click "start review" and do the rest comment.

After comment finish (maybe multiple lines), the top nav bar has a green button "Review changes" with total comment count, click it and add the summary comment for this review, and choose this is a comment, approve, or requests changes. In this case, you will choose requests changes for it. Also, Python member's requests changes will block the merge process, until all requests changes were approval.

In summary, in GitHub workflow, code review will be like a request, and the main committer stays for who create this PR, not reviewers.
msg292969 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 08:00
A review is not required to commit.  The PR itself says "Add more commits by pushing to the bpo-19903 branch on lulouie/cpython."  As far as I know, unresolved requests do not block for cpython.  The Merge button appears to still be 'alive' after my red review.

I have done reviews on other issues, both + and -, with comments on individual lines.  I like it best when there is only a single commit at the time of review.  Otherwise, I don't see any way to comment on the total change after multiple unsquashed commits.  I also don't like commenting on an obsolete line or searching through multiple commits to find the last that touched a line.

I accepted your invitation over 1/2 hour ago and got confirmation from github.  But pushing with this revised command
git push pr_1382:bpo-19903
still fails, with the same message.
msg292985 - (view) Author: Louie Lu (louielu) * Date: 2017-05-04 14:57
Thing just getting weird. If we remove the guard, some of the builtin function will get not-so-good signature result:

>>> i.signature(range.__init__)
<Signature (self, /, *args, **kwargs)>
>>> i.signature(list.append)                                                                                          
<Signature (self, object, /)>
>>> i.signature([].append)                                                                                            
<Signature (object, /)>

You can check my latest commit, The most big change is about __method__, it will make some different behavior:

str(i.signature(list.__new__)) -> (*args, **kwargs)
str(i.signature(p.sub))        -> (repl, string, count=0)

both of them are BuiltinFunctionType, and the first one in old version will be guard, so the get_argspec result will be docstring, we can't different them in non-guard version.
msg293008 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 19:44
Let's back up.  The high-level specification for get_argspec is something like 'Return information that will help programmers write a correct call."  The proposed implementation strategy is to combine signature info from signature (replacing getfullargspec) and the initial part of the callable docstring.

This goal and strategy mimic help(callable).  So we can use its output as a guide, but improve on it when we can and think we should.  This issue could be framed as 'catch up with help(callable)' which already switched to signature.

The test examples constitute a low-level specification by example.  As such, they should be discussed here before changing the code. Let's consider the ones you listed, using 'help(callable)' as a guide.

>>> help(range.__init__)
Help on wrapper_descriptor:
__init__(self, /, *args, **kwargs)
    Initialize self.  See help(type(self)) for accurate signature.

Not very helpful, but if one types 'range.__init__(', one currently sees the last line and should see the last two lines in the future.

However, the calltip for 'range(' should not be the above.  When type(__init__).__name__ == 'wrapper_descripter, the "fob = ob.__init__" replacement should not happen, at least not until we see a case where a wrapper_descripter has the real signature.  (Whether is it still needed for python-coded classes is a separate issue.)  I don't know if all built-in inits 

>>> help(list.append)
Help on method_descriptor:
append(self, object, /)
    Append object to the end of the list.
>>> help([].append)
Help on built-in function append:
append(object, /) method of builtins.list instance
    Append object to the end of the list.

The signature output is fine with me.  I want future calltips to include it, along with the docstring line.

The only issue is the ', /'.  If people can survive it presence in help output, ditto for calltips.  But whenever a signature string contains '/', we could add, between signature and docstring lines, the following.
"('/' marks preceding arguments as positional-only.)"  If we do this, the string should be a global '_positional = ...' so it can be used as-is for tests.  I am inclined to try this.

I want to leave the '/' in the signature because there have been multiple issues and forum questions about why the equivalent of [].append(object='a') does not work.  Now, some built-in parameters are positional-only, some keyword-only, and some both.  The hint should make this clear.
msg293010 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 20:04
>>> help(list.__new__)
Help on built-in function __new__:

__new__(*args, **kwargs) method of builtins.type instance
    Create and return a new object.  See help(type) for accurate signature.

'list.__new__(' currently pops up just the docstring.  I think adding (*args, **kwargs) on top of it does not hurt and maybe is a slight improvement, as it explains the second line a bit.  In any case, explicitly calling __new__ and __init__ is extremely rare, especially for beginners.

For rexep.sub, adding (repl, string, count=0) on top of "Return the string obtained by replacing the leftmost non-overlapping occurrences o..." is a pure win.  Wrapping the too-long docstring line is a separate matter.
msg293011 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-04 20:25
list.__new__ is inherited from object. Look at list.__init__.
msg293012 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 20:29
help(object.__init__) and help(list.__init__) have exactly the same output.
msg293013 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-04 20:50
Ah, yes, __new__ and __init__ differ from ordinal methods. They don't have docstrings generated by Argument Clinic. But list itself as a callable have correct signature.

>>> inspect.signature(list)
<Signature (iterable=(), /)>
msg293019 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 21:34
(I presume'ordinal' meant 'ordinary'.)  I don't know where signature finds the info on built-in type objects   Methods like .append have .__text_signature__.  List does not, and list.__call__.__text_signature is the generic '($self, /, *args, **kwargs)'.  That signature finds it somewhere is a reason for the switch. There is no longer a signature in the first lines of the docstring. So currently, 'list(' only displays "Built-in mutable sequence."

Louie, I verified that for python-coded classes, signature itself gets the info from the __init__ method, so we don't need 'fob = ob.__init__' for python classes either.
msg293020 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-04 21:41
>>> list.__text_signature__
'(iterable=(), /)'

But IDLE shouldn't use __text_signature__ directly, it should use the inspect module.
msg293024 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-04 22:34
Interesting  dir(list.append) and the IDLE completion box show __text_signature__.  dir(list) and the box do not.
msg293042 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-05-05 05:01
As well as __name__, __qualname__, __module__, __bases__, __call__, mro, etc.
msg293052 - (view) Author: Louie Lu (louielu) * Date: 2017-05-05 07:16
But somehow in this case, List have no __text_signature__:

>>> str(i.signature(list))
'(iterable=(), /)'
>>> list.__text_signature__
'(iterable=(), /)'
>>> str(i.signature(List))
'(iterable=(), /)'
>>> List.__text_signature__
msg293055 - (view) Author: Louie Lu (louielu) * Date: 2017-05-05 07:33
And, another case, the __text_signature__ and i.signature() is similar, but not the same:

>>> range.__init__.__text_signature__
'($self, /, *args, **kwargs)'
>>> str(i.signature(range.__init__))
'(self, /, *args, **kwargs)'

So we can't simply compare-and-eliminate this case.
msg293083 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-05-05 09:20
What is List?  Not a builtin.

I am guessing that $self means to remove for bound methods.  In any case, __text_signature__ is effectively private to inspect.signature.  We only care what the latter produces.
msg300041 - (view) Author: Vedran Čačić (veky) * Date: 2017-08-10 04:23
Am I right in assuming this will also fix the bug mentioned in this comment

? If yes, I'm glad I didn't have to report it. :-)
msg300042 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-10 04:46
New changeset 646f6c3096abfe5bde13f039ebf32bce5baf083a by Terry Jan Reedy in branch '3.6':
[3.6] bpo-19903: IDLE: Calltips changed to use inspect.signature (GH-2822) (#3053)
msg300045 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-10 04:59
Although my freshly opened window says PR2822 is open, it was merged with
4 hours ago, and then backported.  Louie, thanks for the patch and the persistence.
msg300047 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-10 05:35
This issue is about IDLE getting further out of the business of calculating signatures and depending instead on the newest (and tested) inspect function.

In 2012, the blogger used getargspec, which was deprecated for 3.x a few years before, in 3.0.  I suspect that the blog is about 2.x, which is no longer relevant for IDLE development.

Inspect.getargspec was superceded by getfullargspec, which was superceded by signature.  Read more at

Signature() has a keyword-only option follow_wrapped=True.  The new patch uses this default.  I suspect that this is relevant to the blogger's concern, but I don't know which value the blogger would want.  Either way, I would not switch unless convinced that False is the best choice for beginning users.  Any such discussion would be a new issue.
msg300048 - (view) Author: Vedran Čačić (veky) * Date: 2017-08-10 05:40
Hm... now I see that link is very misleading... it opens a blog post, and then only a few seconds later jumps to the comment. So now I'll paste the comment here, to avoid misunderstandings:

Python3.6.0 (now in 2017). I start IDLE, and write

>>> def deco(f):
        import functools
        def ret(*args, **kw):
                return f(*args, **kw)
        return ret

>>> @deco
def f(n, k):
        return n + k*2

>>> f(

And get a tooltip (*args, **kw) instead of (n, k). I guess this is a bug.
msg300049 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-08-10 06:15
This is much clearer.  What you got is a limitation of getfullargspec relative to signature (see the links).  In this case, you want follow_wrapped=True, to get the specific signature of the wrapped function instead of the generic signature of the wrapper*. With the patch, the tip in 3.6 and 3.7 is '(n, k)'.  So yes, this is one of the changes consequent on the switch.  

>>> str(inspect.signature(f,follow_wrapped=False))
'(*args, **kw)'
Date User Action Args
2017-08-10 06:15:00terry.reedysetmessages: + msg300049
2017-08-10 05:40:46vekysetmessages: + msg300048
2017-08-10 05:35:07terry.reedysetmessages: + msg300047
2017-08-10 04:59:32terry.reedysetstatus: open -> closed
resolution: fixed
messages: + msg300045

stage: commit review -> resolved
2017-08-10 04:46:31terry.reedysetmessages: + msg300042
2017-08-10 04:23:16vekysetnosy: + veky
messages: + msg300041
2017-08-10 01:18:32terry.reedysetpull_requests: + pull_request3086
2017-08-04 02:53:30terry.reedysetstage: test needed -> commit review
2017-07-23 08:53:50louielusetpull_requests: + pull_request2874
2017-05-05 09:20:43terry.reedysetmessages: + msg293083
2017-05-05 07:33:38louielusetmessages: + msg293055
2017-05-05 07:16:56louielusetmessages: + msg293052
2017-05-05 05:01:49serhiy.storchakasetmessages: + msg293042
2017-05-04 22:34:41terry.reedysetmessages: + msg293024
2017-05-04 21:41:45serhiy.storchakasetmessages: + msg293020
2017-05-04 21:34:42terry.reedysetmessages: + msg293019
2017-05-04 20:50:49serhiy.storchakasetmessages: + msg293013
2017-05-04 20:29:54terry.reedysetmessages: + msg293012
2017-05-04 20:25:08serhiy.storchakasetmessages: + msg293011
2017-05-04 20:04:17terry.reedysetmessages: + msg293010
2017-05-04 19:49:18yselivanovsetnosy: - yselivanov
2017-05-04 19:44:35terry.reedysetmessages: + msg293008
2017-05-04 14:57:19louielusetmessages: + msg292985
2017-05-04 08:00:22terry.reedysetmessages: + msg292969
2017-05-04 06:21:13louielusetmessages: + msg292966
2017-05-04 05:59:39terry.reedysetnosy: + serhiy.storchaka
messages: + msg292964
2017-05-03 09:15:17terry.reedysetmessages: + msg292864
2017-05-02 17:05:52terry.reedysetmessages: + msg292787
2017-05-02 10:31:07louielusetmessages: + msg292741
2017-05-02 10:28:20louielusetpull_requests: + pull_request1490
2017-05-02 07:00:39louielusetmessages: + msg292721
2017-05-02 06:47:08louielusetmessages: + msg292719
2017-05-02 02:56:54terry.reedysetmessages: + msg292712
2017-04-30 17:06:31louielusetnosy: + louielu
messages: + msg292636
2017-02-25 22:55:51terry.reedysetmessages: + msg288586
versions: + Python 3.6, Python 3.7, - Python 3.5
2017-02-25 22:50:10terry.reedylinkissue29653 superseder
2015-02-01 10:12:07terry.reedysetmessages: + msg235157
2015-02-01 09:50:26terry.reedysetmessages: + msg235156
2015-02-01 09:30:15terry.reedysetmessages: + msg235154
2014-12-10 00:52:06yselivanovsetversions: - Python 3.4
2014-05-27 05:53:14terry.reedysetmessages: + msg219203
2014-03-17 17:31:04terry.reedylinkissue1350 dependencies
2014-03-17 17:17:58terry.reedysetmessages: + msg213878
versions: + Python 3.5, - Python 3.3
2014-02-25 00:57:45yselivanovsetnosy: + yselivanov
2014-01-27 02:40:14terry.reedysetdependencies: + inspect.signature does not support new functools.partialmethod, inspect.signature removes initial starred method params (bug)
2014-01-22 08:32:31terry.reedysetmessages: + msg208757
2014-01-04 16:28:05serhiy.storchakasetdependencies: + Move CallTips tests to idle_tests
2013-12-06 00:21:46terry.reedycreate