classification
Title: pydoc help (or help('help')) should show the doc for help
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: merwok Nosy List: Cubky, CuriousLearner, Devin Jeanpierre, ezio.melotti, jairotrad, jaysinh.shukla, jbitcm-, merwok, mikehoy, ncoghlan, r.david.murray, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2012-01-01 08:22 by Devin Jeanpierre, last changed 2017-06-08 10:50 by ncoghlan.

Files
File name Uploaded Description Edit
r74214.diff Devin Jeanpierre, 2012-01-01 08:22 review
i13691.patch Cubky, 2012-06-10 14:01 review
Pull Requests
URL Status Linked Edit
PR 172 closed CuriousLearner, 2017-02-19 07:32
Messages (28)
msg150427 - (view) Author: Devin Jeanpierre (Devin Jeanpierre) * Date: 2012-01-01 08:22
What follows is a copy-paste of a shell session. Notice that at the end, rather than being inside the online help utility, I'm still in the interactive interpreter. I was able to duplicate this on python3.2, python2.7, and python2.6 (verifying it on other versions would have required installing them). Reading the source in trunk, there is nothing that looks like it actually should run this interactive help session. It's just missing.

I guess nobody used this, eh? I've attached a patch that should fix it. I'm not sure how you want to handle adding a test for this, so please advise me on that.

-----

>>> help('help')

Welcome to Python 3.2!  This is the online help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics".  Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".

>>>
msg150750 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-01-06 17:11
IMO, help('help') should document the help function, not start an interactive help session (that’d be help()).
msg150778 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-01-07 00:22
I agree. It should explain the three options:
help(object): help on object or class of object, except
help('name'): help on object/module named 'name'
help(): run utility, which starts with utility help

help(help) prints unhelpful
"Help on _Helper in module site object:

class _Helper(builtins.object)
 |  Define the builtin 'help'.
 |  This is a wrapper around pydoc.help (with a twist).
 |  
 |  Methods defined here:
...
"
which says not at all how to use help().

I have the feeling that there is a related issue on the tracker, but searching for 'help', 'help()', or 'help(help)' all return 620 hits.
msg150783 - (view) Author: Devin Jeanpierre (Devin Jeanpierre) * Date: 2012-01-07 02:17
> IMO, help('help') should document the help function, not start an interactive help session (that’d be help()).

Ahhh, that explains it. help('help') isn't ever meant to be called; it's supposed to be:

>>> help()
...
help> help
...

the call to "help" at the help> prompt is equivalent to help('help').


help('help') should do something different, such as say how to use the help function, as you say. Whereas "help> help" makes sense.
msg150799 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-01-07 14:48
I’m interested in this, so I’m taking assignment.  If a contributor is interested in making a patch I’ll review it, otherwise I’ll say when I start on a patch.
msg151335 - (view) Author: (jbitcm-) Date: 2012-01-16 07:20
I am working on it
msg151463 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-01-17 16:13
Just a heads-up: I’ll be offline between January 19 and the end of the month, so don’t worry if you make a patch and it’s not reviewed immediately (at least not by me, other developers may do it :)
msg162585 - (view) Author: Petr Kubat (Cubky) Date: 2012-06-10 14:01
Is anyone still working on this? If not I would like to make this work.
Although I'm not exactly sure how to tackle this problem since the built-in help function is defined as a wrapper around pydoc.help and so the docstring used for printing help('help') is actually a docstring of the site._Helper() class.
I made a patch that adds a documentation-like message to the pydoc.help function when it is called with a 'help' argument, but it doesn't seem clean to me.
msg162590 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-10 17:12
It doesn't look like anyone is working on it currently.

Yes, getting this right is a bit tricky for exactly the reason you say.  You have to make sure you don't break the other uses, and glancing at the patch (I haven't looked at the code you are patching) it seems likely that the change to the if statement is going to break *something*.  Probably what happens when you type 'help' at the 'help>' prompt, at a guess.
msg162593 - (view) Author: Petr Kubat (Cubky) Date: 2012-06-10 18:47
Help at the help> prompt does work, I tested that. But if you (or anyone) thought of a better way to fix this issue I would be glad to change it.
msg162595 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-10 20:38
"works" in the sense that it produces output.  But it doesn't produce the output it used to, which is the same text as the initial banner when you type help().  I believe the existing behavior in this case is correct.  With your patch it produces the same output as "help('help')" at the python prompt does, and what that output describes doesn't work at the 'help>' prompt.

Thanks for tackling this.

Hmm.  How about adding the special case to _Helper.__call__?
msg162597 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-06-10 20:41
Oh, that wouldn't fix 'pydoc help', would it.  So there are probably two places where the special case has to be added, which means the code itself should go in pydoc and be called from _Helper.__call__ somehow.  (Note, I'm just throwing out ideas here, I haven't looked carefully at the code.)
msg162608 - (view) Author: Petr Kubat (Cubky) Date: 2012-06-11 09:29
I see. So calling help('help') should produce the documentation on the help() function and typing help at the help> prompt should print the help for the prompt.
Tricky indeed. I think I'll look at it during the day after tomorrow and post some results (if any).
msg172356 - (view) Author: Mike Hoy (mikehoy) * Date: 2012-10-08 02:21
I imported this patch on 3.4 and it worked as expected on my system.

help('help') brings up info about help()
help(object) prints the docstring
help() invokes the help
msg172358 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-10-08 02:35
What about "pydoc help"?
msg172359 - (view) Author: Mike Hoy (mikehoy) * Date: 2012-10-08 03:29
>> What about "pydoc help"?

$ ./python -m pydoc help

Help on built-in function help:

help(...)
Invoke the built-in help system.

    help()
    The interactive help system starts on the interpreter
    console.

    help(string)
    The string is looked up as the name of a module, function, class, method,
    keyword, or documentation topic, and a  help page is printed on the
    console.   

    help(object)
    Generates a help page on the given object.

Or did you mean inside the interpreter?
msg172593 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-10-10 16:25
Cool!  Can you add tests?
msg172603 - (view) Author: Mike Hoy (mikehoy) * Date: 2012-10-10 21:24
>>> Cool!  Can you add tests?

Nope, I can't add tests.
msg172605 - (view) Author: Éric Araujo (merwok) * (Python committer) Date: 2012-10-10 21:57
I was addressing the author of the patch :)
msg248686 - (view) Author: Jairo Trad (jairotrad) * Date: 2015-08-16 18:31
I just tested this issue in Python 3.6.0a0 and got this behavior:

help('help') brings 

"Help on _Helper in module _sitebuiltins object:"

help(help) brings:
Help on _Helper in module _sitebuiltins object:

help() invokes the help command line.

So this was fixes on the way to python3.6, I can make the test for 3.4 but is 3.4 closed for bug fixing?

Can someone check this in 3.5?
msg248694 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2015-08-16 22:53
3.5 is the same as 3.6 in that now both help(help) and help('help') show the not terribly helpful "Help on _Helper in module site object:...". Buried in the text is  
 |  Calling help() at the Python prompt starts an interactive help session.
 |  Calling help(thing) prints help for the python object 'thing'.

As before, I think the _Helpter stuff should be eliminated and 
explanation added for help('topic') and a description of what 'topic's are specially recognized.

There has been this improvement
>>> help('jslfjslfdjlskfj')  # 3.4
no Python documentation found for 'jslfjslfdjlskfj'

>>> help('jsflksjflkjsl')  # 3.5
No Python documentation found for 'jsflksjflkjsl'.
Use help() to get the interactive help utility.
Use help(str) for help on the str class.

but the latter omits decription of what 'topic' will give special output.  By experiment, keywords (help('if')), builtins, and unimported modules (help('itertools')) works.

Yes, 3.4 can be patched.
msg272588 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-08-13 14:09
No 3.4 can't be patched, this is not a security issue.  A patch to make the behvior what we want should be generated against either 3.5 or 3.6, and the committer will deal with any issues arising from 3.5 vs 3.6 issues.
msg288109 - (view) Author: Sanyam Khurana (CuriousLearner) * Date: 2017-02-19 05:57
Hi, I'm working on fixing this issue.
msg288112 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-19 06:27
Recapping the situations that need test cases before this can be merged:

* behaviour when "help" is entered at the interactive help prompt (as the current behaviour is correct and should *not* change)
* behaviour when calling "help(help)"
* behaviour when calling "help('help')"
* behaviour when running "python3 -m pydoc help"

In the latter 3 cases, the default site._Helper docs currently shown are not particularly helpful.

It does make me wonder whether it might be worth defining a __help__ magic method that completely overrides what help(obj) displays. It would limit the fix to 3.7+, but it would make it possible to do this cleanly just by defining __help__ on site._Helper.
msg291365 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-04-09 09:47
After reviewing Sanyam's PR at https://github.com/python/cpython/pull/172 I ended up rejecting it as an approach to resolving this issue.

The core problem with the approach is that accessing `_sitebuiltins._Helper` is an implementation detail twice over:

- Python implementations aren't required to have a _sitebuiltins module
- that module isn't required to contain a _Helper class

That means importing it into a general purpose module like pydoc would couple pydoc specifically to the way CPython works, which is something we try to avoid doing.

By contrast, a richer `__help__()` protocol to override the `__doc__` based default behaviour would allow any object to implement it, and _sitebuiltins._Helper could just use that protocol to special case itself.
msg291371 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-04-09 10:47
> It does make me wonder whether it might be worth defining a __help__ magic method that completely overrides what help(obj) displays.

This should be very complex protocol if take to account that help() outputs formatted and highlighted text and pydoc can output to html format, and alternative Python shells can provide alternative help().
msg291372 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-04-09 10:56
Aye, it's definitely a tricky problem. While I didn't take them into account in my review, I now realise those issues also impacted the attempted workaround - I'd be surprised if it worked as intended in those other contexts (even the "pydoc -b" rendered HTML view).

As far as how to deal with the rendering complexity in a protocol, one possible way to go would be to say that rather than returning a fully rendered string, __help__() methods should return either a Python function object or a type() instance (that doesn't define __help__()), and then pydoc would render the help based on that.

Regardless of the exact approach, it would be a task requiring a PEP to resolve, since rendered documentation appears in so many different contexts.
msg295401 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-06-08 10:50
Belatedly removing the "easy" tag, since that turned out to be thoroughly incorrect...
History
Date User Action Args
2017-06-08 10:50:22ncoghlansetkeywords: - easy

messages: + msg295401
2017-04-09 10:56:08ncoghlansetmessages: + msg291372
2017-04-09 10:47:53serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg291371
2017-04-09 09:47:46ncoghlansetmessages: + msg291365
2017-02-19 07:53:54jaysinh.shuklasetnosy: + jaysinh.shukla
2017-02-19 07:32:52CuriousLearnersetpull_requests: + pull_request138
2017-02-19 06:27:21ncoghlansetnosy: + ncoghlan

messages: + msg288112
versions: + Python 3.7, - Python 3.5
2017-02-19 05:57:23CuriousLearnersetnosy: + CuriousLearner
messages: + msg288109
2016-08-13 14:09:14r.david.murraysetmessages: + msg272588
versions: + Python 3.5, Python 3.6, - Python 2.7, Python 3.2, Python 3.3, Python 3.4
2016-08-13 10:46:35SilentGhostsetnosy: + terry.reedy, Devin Jeanpierre, ezio.melotti, merwok, r.david.murray, mikehoy, jbitcm-, Cubky, jairotrad, - lissacoffeyx
2016-08-13 10:45:37SilentGhostsetmessages: - msg272582
2016-08-13 10:36:43lissacoffeyxsetnosy: + lissacoffeyx, - terry.reedy, Devin Jeanpierre, ezio.melotti, merwok, r.david.murray, mikehoy, jbitcm-, Cubky, jairotrad
messages: + msg272582
2015-08-16 22:53:03terry.reedysetmessages: + msg248694
2015-08-16 18:31:16jairotradsetnosy: + jairotrad
messages: + msg248686
2012-10-10 21:57:37merwoksetmessages: + msg172605
2012-10-10 21:24:48mikehoysetmessages: + msg172603
2012-10-10 16:25:09merwoksetmessages: + msg172593
stage: needs patch -> test needed
2012-10-08 03:29:47mikehoysetmessages: + msg172359
2012-10-08 02:35:34merwoksetmessages: + msg172358
versions: + Python 3.4
2012-10-08 02:21:45mikehoysetmessages: + msg172356
2012-10-08 02:13:07mikehoysetnosy: + mikehoy
2012-06-11 09:29:50Cubkysetmessages: + msg162608
2012-06-10 20:41:01r.david.murraysetmessages: + msg162597
2012-06-10 20:38:27r.david.murraysetmessages: + msg162595
2012-06-10 18:47:20Cubkysetmessages: + msg162593
2012-06-10 17:12:40r.david.murraysetnosy: + r.david.murray
messages: + msg162590
2012-06-10 14:01:53Cubkysetfiles: + i13691.patch
nosy: + Cubky
messages: + msg162585

2012-01-17 16:13:50merwoksettitle: pydoc help (or help('help')) claims to run a help utility; does nothing -> pydoc help (or help('help')) should show the doc for help
2012-01-17 16:13:09merwoksetmessages: + msg151463
2012-01-16 07:20:36jbitcm-setnosy: + jbitcm-
messages: + msg151335
2012-01-07 14:48:19merwoksetkeywords: + easy
assignee: merwok
messages: + msg150799
2012-01-07 03:52:34ezio.melottisetnosy: + ezio.melotti

type: behavior
stage: needs patch
2012-01-07 02:17:17Devin Jeanpierresetmessages: + msg150783
2012-01-07 00:22:00terry.reedysetnosy: + terry.reedy
messages: + msg150778
2012-01-06 17:11:10merwoksetnosy: + merwok

messages: + msg150750
versions: + Python 3.3, - Python 2.6
2012-01-01 08:22:31Devin Jeanpierrecreate