classification
Title: Add introspection information for builtins
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: larry Nosy List: barry, brett.cannon, georg.brandl, gvanrossum, larry, ncoghlan, python-dev, serhiy.storchaka
Priority: normal Keywords:

Created on 2013-11-20 23:52 by larry, last changed 2013-11-23 23:39 by larry. This issue is now closed.

Files
File name Uploaded Description Edit
larry.introspection.for.builtins.patch.1.txt larry, 2013-11-20 23:52 Rough patch with a first effort. review
larry.introspection.for.builtins.patch.2.txt larry, 2013-11-21 21:09 review
larry.introspection.for.builtins.patch.3.txt larry, 2013-11-22 22:57 review
larry.introspection.for.builtins.patch.4.txt larry, 2013-11-23 22:39 review
Messages (16)
msg203549 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-20 23:52
Let's see if we can get introspection information for builtins using the Clinic for 3.4.

Georg suggested part of the approach while we were hanging out in Tokyo.  I'd considered it previously before but dispensed with the idea because it seemed too loopy.  Actually it seems to work great.  

The approach:

* Clinic generates an extra first line for the docstring, that looks like "def (...)\n".  (Note: no function name!)

* The PyCFunctionObject __doc__ getter detects this line and skips it if present.

* Add a new getter to PyCFunctionObject, which I've called "__textsig__", that returns this first line if present in the docstring.  (It skips the "def " at the front, and clips it at the newline.)

* inspect now notices if it's passed in a PyCFunctionObject.  If it gets one, it checks to see if it has a valid __textsig__.  If it does, it parses it (using ast.parse) and produces a valid signature.

Advantages of this approach:
* It was easy to do and took very few lines of code.

* For signatures that are impossible to convert to Clinic, we can
  write the metadata by hand.

Disadvantages of this approach:
* Uh, nothing, really!

The next step is probably to convert pydoc to use inspect.signature instead of the manky old methods it currently uses.  After that, clean up the patch, and add a unit test or two.

What do you think?
msg203550 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-20 23:59
If you review, you can ignore the changes to the .c files, those only got touched because of the new autogenerated "def (" lines in the docstrings.  No other changes there.
msg203551 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-21 00:00
(Well, except for Object/methodobject.c.)
msg203575 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2013-11-21 05:45
I think there's a similar but slightly different convention in Sphinx
autodoc -- the first line of the docstring can be "name(...)" . Isn't that
better than def?
msg203576 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-21 05:50
I was deliberately trying to avoid something that a person might do by accident.  Also, "def (" is consistently only five bytes, whereas "pterodactyl (" or whatever will often be much longer, in case static data size is a concern.

But I could switch it to that if you think that's better.  Certainly it's a tiny amount more readable.

(I don't think this would make the docstrings compatible with sphinx autodoc though.)
msg203583 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-11-21 08:12
Great! See also issue16842 and issue16638.
msg203607 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-21 10:36
Sounds good to me (with either Larry's or Guido's spelling).

We should ensure this still works with the config option that disables docstrings entirely.
msg203667 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-21 19:12
For Nick: "still works" as in the metadata is still available without docstrings or "still works" as in it won't crash?
msg203672 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-21 21:09
I changed it to "<callablename>(".  Patch attached.  Still works.
msg203684 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-21 22:14
"still works" as in "doesn't crash and the docstrings are still missing to
save memory".
msg203797 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-22 16:27
Anybody have a better name for __textsig__ ?
msg203827 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-11-22 18:16
__signature_text__? __textsignature__? No need to abbreviate that much since there is no built-in function equivalent and people are not expected to work with it.
msg203908 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-22 22:57
I went with __text_signature__.  I also did some more polishing, and added / fixed the unit tests, and even added a Misc/NEWS.  Is it good enough to go in before the beta?
msg204026 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2013-11-23 14:19
+1 from me (I see this as the key benefit of argument clinic).

For 3.5, we can look at ducktyping on the attribute, but for now I think
it's worth keeping that text format internal to CPython.
msg204120 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-11-23 22:39
Fresh patch attached.  pydoc now uses inspect.signature instead of inspect.getfullargspec to generate the arguments for the function, and supports builtins.  That's everything :D

Planning on checking this in pretty soon, to get it in for beta (which hopefully still gets tagged today).
msg204129 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-23 23:38
New changeset 78ec18f5cb45 by Larry Hastings in branch 'default':
Issue #19674: inspect.signature() now produces a correct signature
http://hg.python.org/cpython/rev/78ec18f5cb45
History
Date User Action Args
2014-02-25 00:27:31yselivanovlinkissue17053 superseder
2013-11-23 23:39:10larrysetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2013-11-23 23:38:33python-devsetnosy: + python-dev
messages: + msg204129
2013-11-23 22:39:59larrysetfiles: + larry.introspection.for.builtins.patch.4.txt

messages: + msg204120
2013-11-23 14:19:17ncoghlansetmessages: + msg204026
2013-11-22 22:57:47larrysetfiles: + larry.introspection.for.builtins.patch.3.txt

messages: + msg203908
2013-11-22 18:16:47brett.cannonsetmessages: + msg203827
2013-11-22 16:27:36larrysetmessages: + msg203797
2013-11-21 22:14:15ncoghlansetmessages: + msg203684
2013-11-21 21:09:29larrysetfiles: + larry.introspection.for.builtins.patch.2.txt

messages: + msg203672
2013-11-21 19:12:01brett.cannonsetmessages: + msg203667
2013-11-21 14:38:27larrysetnosy: + barry
2013-11-21 10:36:25ncoghlansetmessages: + msg203607
2013-11-21 08:12:59serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg203583
2013-11-21 05:50:40larrysetmessages: + msg203576
2013-11-21 05:45:00gvanrossumsetmessages: + msg203575
2013-11-21 05:35:28larrysetnosy: + ncoghlan
2013-11-21 00:00:08larrysetmessages: + msg203551
2013-11-20 23:59:24larrysetmessages: + msg203550
2013-11-20 23:52:26larrycreate