classification
Title: Modernize pydoc to use better HTML and separate CSS
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Frédéric Jolliton, Mariatta, PeterLovett, aroberge, azsorkin, belopolsky, berker.peksag, donmez, eric.araujo, ezio.melotti, georg.brandl, lukasz.langa, mdk, r.david.murray, rhettinger, ron_adam, serhiy.storchaka, terry.reedy, tritium, vterron, xtreak
Priority: normal Keywords: patch

Created on 2010-12-16 09:25 by rhettinger, last changed 2019-03-24 05:38 by xtreak.

Files
File name Uploaded Description Edit
css_v2.diff ron_adam, 2011-01-02 07:42 css support in the new pydoc pages. review
css_v3.diff ron_adam, 2011-01-02 22:19 css support for the new pydoc pages. review
css_master.diff xtreak, 2019-03-24 05:35
Screen Shot 2019-03-24 at 10.58.33 am.png xtreak, 2019-03-24 05:35 pydoc with updated CSS and bootstrap
Messages (51)
msg124119 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-12-16 09:25
This is a straight-forward project.

Pydoc currently generated 1990's style html which mixes content and presentation, making it very difficult for users to customize the appearance of the output.

It is full of html like:

<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">
<tr bgcolor="%s">
<td valign=bottom>&nbsp;<br>
<font color="%s" face="helvetica, arial">&nbsp;<br>%s</font></td
><td align=right valign=bottom
><font color="%s" face="helvetica, arial">%s</font></td></tr></table>
    ''' % (bgcol, fgcol, title, fgcol, extras or '&nbsp;')

    def section(self, title, fgcol, bgcol, contents, width=6,
                prelude='', marginalia=None, gap='&nbsp;'):
        """Format a section with a heading."""
        if marginalia is None:
            marginalia = '<tt>' + '&nbsp;' * width + '</tt>'
        result = '''<p>
<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="section">
<tr bgcolor="%s">
<td colspan=3 valign=bottom>&nbsp;<br>
<font color="%s" face="helvetica, arial">%s</font></td></tr>
    ''' % (bgcol, fgcol, title)
        if prelude:
            result = result + '''
<tr bgcolor="%s"><td rowspan=2>%s</td>
<td colspan=2>%s</td></tr>
<tr><td>%s</td>''' % (bgcol, marginalia, prelude, gap)

Please convert it to simple, validated HTML with the formatting moved to a simple, validated default style sheet.  Liberally apply div/span elements with class/id attributes as necessary.
msg124144 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-16 18:20
I believe Ron has done something along these lines already.  See issue2001.  Committed css file is empty, (see r86971) but I am adding Ron to "nosy" in case he has something in the pipeline.
msg124147 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-12-16 18:22
See also the calendar module for an example of a "modern" html.
msg124157 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-16 21:12
I uploaded the css file I used in an experimental version of pydoc.  It may give some useful starting values. 

Before this is done, the old server code should be removed (also for 3.3). (another issue?)  There are two files in the tools/scripts directory that may need attention as well. pydocgui.pyw and pydoc3.
msg124158 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-16 21:17
Those two scripts are just three-liners pydoc launchers.

I disagree about the CSS reset.  Default values and user-set values have a reason for being there, and I share the viewpoint that they should not be reset.
msg124159 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-12-16 21:42
I'm looking for a deeper fix, all the in-line styling replaced by a stylesheet.  Can you guys work together on bring this to fruition?
msg124162 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-16 22:40
Eric, most of what's in that file is what I figured out by trial and error in order to get it to work on the different browsers at that time. (about 3 years ago.)  You are probably more experienced with css than I am, so you are more than welcome to update and change anything in there. :-)

What do you think about starting with a set of static html pages to get the css and html to work nice, and then only after that is done, edit pydoc to generate those pages.  That should get us mostly there, and simplify what needs to be done in pydoc.
msg124164 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-12-16 23:34
A good procedure is to start afresh with an empty, embedded style sheet and replace the html styling attributes one at a time, while keeping the overall look and feel the same.   At the end of the process the embedded style sheet can be converted to external.
msg124176 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-17 03:06
I think that's how I ended up with the style sheet I uploaded.  It works, but it can be a slow process.

Another factor is the pydoc server will reread an external style sheet on browser refreshes. So you can see the results of style sheet changes without restarting pydoc. If it's embedded, you need to edit the program and restart everything each time.

I was thinking that I may still be able to make a set of static html files that go along with that style sheet.  Then we can adjust the css class names and make other changes, then use that as a guide for replacing the html.

Eric, what do you think?
msg124177 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-12-17 03:32
The uploaded stylestyle is *much* bigger than I expected.  There should be no more than a one-to-one correspondence.  With inheritance, perhaps even fewer entries.
msg124179 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-17 03:41
Ok, I just looked at them again, I didn't remember how different it was.  They probably won't be much help other than maybe seeing how some things could be done. Here's a zip file of some saved pages, so you can take a look.
msg124181 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-12-17 03:42
The CSS also has a validation error, some warnings and could be written better IMHO (specifically I don't like the placement of the {}).

I also don't believe that extensive CSS resets are useful -- they usually just lead to lot of redefining. I usually limit myself to * { margin: 0; padding: 0; } and possibly specify specific values when necessary. Minor differences with different browsers can be perfectly fine in lot of cases.
msg124195 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-17 11:47
[Raymond]
> I'm looking for a deeper fix, all the in-line styling replaced by a
> stylesheet.  Can you guys work together on bring this to fruition?
When I talked about the CSS reset, I was referring to a precise part of the file proposed by Ron, so we’re already discussing together :-)

I wonder how desirable it is to preserve the look and feel of the pages.  We all agree on externalizing the CSS and add a way for the users to add their own stylesheet; why not take the opportunity to also improve the style?  Huge blocks of colors are not that attractive to me :)

Regarding workflow, I’d find easier to start from bare HTML that works nice (yes, I test with w3m) and then add style.  Technically, I’d like to maintain the HTML as a small set of files (in pydoc_data) containing fragments of HTML with placeholders ({} or $): That’s easy to edit, to validate (a very simple script can put the fragments together) and to use.

I agree that the CSS file should be as short as possible (in content), but not too short (in file size).  For example, trailing commas in properties and brackets on their own line will allow nice diffs, just like Python.

(I won’t have much time for Python in December, but we have a lot of time before 3.3b1 :)
msg124306 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-12-18 18:43
There's no rush on this one.
There desired properties are:
* separate content from presentation
* validated CSS and HTML
* simple CSS so that people can *easily* customize it
* possibly offer two premade stylesheets
  1) a default beautiful one
  2) one the closely approximates the current look and feel

If the current look cannot be recreated, it should be taken as a sign that the content/presentation split wasn't done well.
msg124825 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-28 22:48
Here is a tentative start on this.  (css_v1.diff)

The css file is much better.  It's shorter, simpler and validated.

The header and navbar panel use it in the new server.

Added a markup call to the topic page contents.  (The same markup call is already used for the doc strings.)

Extra items in the css are what I come up with from testing different ways of doing things.
msg124857 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-29 17:12
The HtmlDoc class has methods that take colors. Can this be changed or does it need to be depreciated first? 

    def heading(self, title, fgcol, bgcol, extras=''):
        """Format a page heading."""
        return '''
<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">
<tr bgcolor="%s">
<td valign=bottom>&nbsp;<br>
<font color="%s" face="helvetica, arial">&nbsp;<br>%s</font></td
><td align=right valign=bottom
><font color="%s" face="helvetica, arial">%s</font></td></tr></table>
    ''' % (bgcol, fgcol, title, fgcol, extras or '&nbsp;')

For the interactive server, I can override these methods with no problem, but the generated docs won't benefit from this until the HtmlDoc class is replaced.  Any suggestions?
msg124858 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-12-29 17:14
Well, you could reuse these arguments to mean CSS classes, and have styles like ".red { color: red }" :)
msg124863 - (view) Author: Ron Adam (ron_adam) * Date: 2010-12-29 18:23
It may be useful to change those to 'id=' and 'class=' if possible.

It isn't clear to me how much of pydoc is still part of the public api in python 3.x.  pydoc.__all__ is set only to ['help'].  

Entering help(pydoc) just gives the basic help and command line arguments along with.

DATA
    __all__ = ['help']
    help = <pydoc.Helper instance>

There is nothing in the official (online) docs on importing pydoc or any of it's classes or methods.  

But dir(pydoc) shows all the methods, and the HTMLDoc class is still importable even though they aren't listed in __all__.
msg125033 - (view) Author: Ron Adam (ron_adam) * Date: 2011-01-02 07:42
Here is a new diff which updates all the new pydoc pages to use the css file. 

The css file is simpler and cleaner.  I also made a few adjustments to the url handler error handling, and changed the titles in the head sections so they say "Pydoc" instead of "Python" as they are PyDoc pages about Python.

None of these changes effect any of the old pydoc code yet.  This is about as far as we can go without removing the old tk panel and server.

Time for some feed back.  And how close do we really need it to be to the original?

:-)
msg125049 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-01-02 13:54
A few comments about css_v2.diff:
1) when the value is '0', there's no need to specify the unit (e.g. 0em);
2) when the color is specified the background-color should be specified as well (and vice versa);
3) hex colors (e.g. #00FF00) should be preferred to named colors (e.g. gray);
4) some selectors and properties don't work with older browsers (e.g. E > F or min-width);
5) there are a few empty dl.*{} that I would remove (unless you plan to fill them later);
6) the style I prefer for CSS is:
selector {
  property: value;
}

Regarding the HTML:
1) using an HTML 4.01 strict doctype would be better;
2) all the style-related attributes and elements should be removed (e.g bgcolor, valign, <font>);
3) using .red { color: red; } is not a good idea. Classes' names should describe the role of the element (e.g. header, entry) and not their style (otherwise when you want to change the page with a blue theme you'll end up with a .red { color: blue; }). If the colors are passed directly to the HTML they should be removed and left to the CSS(s) only. I don't know the code well enough to say if this is doable and/or if it requires a deprecation first;
4) the <li>s in html_header() should be closed, same for all the other elements that support a closing tag, even if optional (e.g. <dt>, <dd>);

There are also some minor incontinences in the indentantion, e.g.:
+        link_list = ['<a href="%s.html">%s</a>' % (name, name)
+                    for name in sys.builtin_module_names
+                    if name != '__main__']
+        contents = [html.index_columns('Built-in modules',
+                    link_list, css_class='section modules')]
+        link_list = ['<a href="%s.html">%s</a>' % (name, name)
+                      for name in sorted(Helper.keywords.keys())]
(the "contents" one is indented correctly), and some extra space after the '('.
msg125111 - (view) Author: Ron Adam (ron_adam) * Date: 2011-01-02 22:19
> If the colors are passed directly to the HTML they should be
> removed >and left to the CSS(s) only. I don't know the code
> well enough to say if this is doable and/or if it requires a
> deprecation first;

We may have to do dome depreciating when it comes to the old HTMLDoc class methods.  Would it be possible to depreciate the whole class instead of the separate methods?  If so, it would be good if we can squeeze that into 3.2. <hint hint>  Or else we may not be able to finish this until Python 3.4.

The empty css elements will be used later.

Here is the css_v3.diff for review.  I'd like to hear any thoughts about the general html structure and class/id names.

Beside validating them, I test with firefox and chromium. I don't have easy access to ms-explorer or the current mac browser.
msg125113 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2011-01-02 22:40
These arguments should not really be of concern.  If we indeed deem them public API, they can stay but be ignored.
msg125122 - (view) Author: Ron Adam (ron_adam) * Date: 2011-01-02 23:15
To go forward I can create a new private api instead of changing HTMLDoc, that would be preferable.

Should the -w option also use the new html pages?  Or do we need a new option for that?
msg127542 - (view) Author: Ron Adam (ron_adam) * Date: 2011-01-30 16:12
A reminder: Check for instances where html.escape is not called on data inserted into the html pages.

I'll update the patch as the non-css (error handling) parts made it into python 3.2.  :-)
msg130629 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-03-11 22:14
I added a few comments for ccs_v3.diff on rietveld.
msg165401 - (view) Author: Víctor Terrón (vterron) * Date: 2012-07-13 19:00
I would like to take on this issue if, as it seems, Ron is no longer working on it. I'm thinking of using HTML5 and maybe replicating the default output of Sphinx.
msg165406 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2012-07-13 19:12
Using an HTML5 doctype is ok, but I'm not sure it's a good idea to use new HTML5 elements/features yet (those shouldn't be necessary anyway, so basically you should end up with valid HTML 4 strict with an HTML5 doctype).
msg183304 - (view) Author: Ron Adam (ron_adam) * Date: 2013-03-02 00:00
I'm going to go over this issue again with fresh eyes after having been away for some time.

Recent experience with another project has helped answer some of the questions I had earlier.  Particulary, how not to over specifying class names and id's.  This should lead to cleaner html pages that will be easier to style with css.  It should also lead to a smaller patch. ;-)

It seems that most of the input so far has more to do with spelling rather than function.  Is there a prefered style guide for css that we should use?
msg183319 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-03-02 12:55
> Is there a prefered style guide for css that we should use?

If you mean indentation/spacing, my preferred style is:

selector {
    property1: value1;
    property2: value2;
    ...
}

FWIW I agree an HTML5 doctype should be used.
msg234936 - (view) Author: Andre Roberge (aroberge) * Date: 2015-01-29 02:04
If anyone is still interested in this, I did that today (scratching a personal itch - not knowing this had been filed before).  

What I have done:
1. Moved all the font/color information to a separate css file
2. Used html5 syntax.
3. Recreated a css style sheet that approximate the current look
4. Created a new, very simple style sheet, that is used as a default. It could stand to be greatly improved upon.
5. Checked the validity (W3C) of a sample of the pages generated. This required me to include some empty <dt> and <dd> tags.
6. Added an option to start the server with an optional, user-defined css stylesheet.
7. Made some small unrelated edit (adding spaces after commas, removing
extra spaces before commas, etc.) to reduce the amount of noise from my linter.

I implemented this as a separate project; I did not attempt to run any existing unit tests, nor create new ones.  Other than for the additional option (user defined style sheet), I tried to avoid making any change to the functionality. The styling possible to do is thus limited as I mostly replaced existing strings/templates by new ones.

My implementation can be found at https://github.com/aroberge/mod_pydoc
(sorry, not an hg repo).  I did not include a license; I took the existing pydoc without asking permission. It can be understood to be under the original license.
msg234940 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-01-29 02:50
Thanks! Apparently, we were working on the same issue simultaneously. I was
working on the HTML and CSS parts (didn't touch to Lib/pydoc.py yet). I can
combine our work, if you could

- create a separate branch (no need to be a Hg repo, I can create a
  patch from GitHub)
- revert whitespace and PEP 8 changes
- revert the new -c option in mod_pydoc.py
msg234941 - (view) Author: Andre Roberge (aroberge) * Date: 2015-01-29 02:58
I could certainly create a new branch and revert the PEP8 changes and the new -c option, but before I do this, could you confirm that the new html output would be deemed to be acceptable?
msg234942 - (view) Author: Andre Roberge (aroberge) * Date: 2015-01-29 03:25
Rather than creating a new branch, I took another copy of pydoc.py, kept its name, and only applied the html-related changes in it. (no new  -c option, no unrelated PEP8 changes to the best of my knowledge.)  The original pydoc.py referred to pydoc_data/_pydoc.css (which was an empty file).  I put the "minimal new styling" option in that file.

I noticed that some former <a name=...> had no href attribute; I changed the "name" attribute to "id", since "name" is deprecated in html 5.  However, my styling option shows these as actual links whereas they are inactive.  An example of this can be seen in _codecs
msg236962 - (view) Author: Ismail Donmez (donmez) * Date: 2015-03-01 13:33
Any update on this? Would be nice to have this for 3.5 release.
msg246917 - (view) Author: Frédéric Jolliton (Frédéric Jolliton) Date: 2015-07-18 20:56
Oh god. The HTML produced by pydoc is awful.

This is absolutely nothing modern about it.

The code itself hurts my brain. It feels very old (14 years old..), and the HTML production is overly complex, and hard to check regarding correct quoting/escaping.

Generating modern HTML5/CSS would mean:

 - Using <h1>, <h2>, for section title,
 - Forget about <br> (this element should never be used nowadays),
 - Forget about <hr> (let the CSS decide when to insert a visual separator after/before a given element),
 - Don't use <code> for block of code. This is an inline element. Use <pre class="code"> for example.
 - Don't use <table> all over the place for formatting everything (use <li> for the list of modules for instance),
 - Drop these useless <dd></dd> (empty!)
 - No need to replace \n by <br>, or to replace space by &nbsp;. The formatting can be achieved by white-space: pre in CSS.
 - <a name=".."> or <a id=".."> could be replaced by <span id=".." class=".."> to distinguish them from hyperlinks.
 - the table "docclass" could be a serie of <h2> (or <h3>) title followed by the content of the section. The table seems useless because there are no particular requirement for an alignment.
 - and so on, and so on, ..

Actually, I think this need a complete rewrite.

This is a useful tool to have included with Python, but this need a serious refresh.

To me, a "modern" documentation is something like this (from the Rust documentation):

    https://doc.rust-lang.org/std/option/enum.Option.html
    https://doc.rust-lang.org/std/option/index.html

(Look at the generated HTML too. It's rather straightforward.)
msg246920 - (view) Author: Alex Walters (tritium) * Date: 2015-07-18 22:53
Isn't this whats sphinx's apidoc is for?
msg277934 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2016-10-03 07:10
(As a result of the discussion at http://psf.upfronthosting.co.za/roundup/meta/issue605, I started to re-triage all easy issues.)

I don't think this is a suitable task for new contributors. It requires a) good HTML and CSS knowledge b) familiarity with pydoc API c) possibly design of a new API to finish this task.

There are also some key points that needs to be discussed:

* The current API ties to the dated HTML output and we may not need some of the members in the new format (look at the signature of HTMLDoc.section() for example) Also, I know HTMLDoc is not a public API, but I saw a number of HTMLDoc subclasses in GitHub (even I did it myself in my own projects) so there is a risk of breaking working applications. My suggestion is to keep the old API as-is and create a new one with new output.

* The another question is that do we need HTMLDoc in 2016? Sphinx, sphinx-autodoc and readthedocs.org doing an amazing job and it might be better to refer people to alternative solutions (like we did for requests) See also https://mail.python.org/mailman/private/core-mentorship/2016-August/003622.html for a discussion about this subject.
msg278084 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2016-10-04 19:57
Thanks for the update, Berker.
Would you be able to remove the 'easy' keyword from this ticket?

I'll keep myself in the nosy list as I'm interested to learn about this anyway :)
msg278089 - (view) Author: Andre Roberge (aroberge) * Date: 2016-10-04 20:46
When rhettinger created this issue, the goal was 

"Pydoc currently generated 1990's style html which mixes content and presentation, making it very difficult for users to customize the appearance of the output. ... Please convert it to simple, validated HTML with the formatting moved to a simple, validated default style sheet."

Apparently, the statement of this issue is no longer correct. However, in order to do so, one apparently needs to have access to a private list mentioned in a comment (https://mail.python.org/mailman/private/core-mentorship/2016-August/003622.html) 

One thing that motivated to submit my tentative "solution" to this issue here almost two years ago (solution which, I thought, met the stated goal) was a discussion at Pycon as to how one could encourage new contributors.  Is this the purpose of the core-mentorship list?  Does one need to join the list if one wishes to contribute?
msg278090 - (view) Author: Mariatta (Mariatta) * (Python committer) Date: 2016-10-04 20:52
Hi Andre,

You can subscribe to the core mentorship mailing list here: 

https://mail.python.org/mailman/listinfo/core-mentorship

Once you subscribed, you'll get access to the said discussion :)

It is not required, but it's a great place to ask questions or get help from existing core developers.

HTH
msg278098 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-05 00:36
I've removed the easy tag.  (Aside for those who can do the operation: to remove all tags from an issue one must select '-no selection-'; just ctrl-clicking the last tag to remove its highlight doesn't work...or at least that's what I remember, I didn't actually test it this time.)
msg278119 - (view) Author: Peter Lovett (PeterLovett) * Date: 2016-10-05 11:15
I see that pydoc still useful, and have 6 points I'd like to work on with pydoc. Is this too much for one issue? Should I break bits out to another issue(s)? Although it would make sense to do it all in one go.

-----------------------------------------------
Pydoc is a good way to get users (especially beginners) used to writing docstrings, and leads nicely into doctest and similar tools.
Other languages, such as Perl and Java have similar tools that generate text or html documentation from code.
Some have used pydoc classes for their own uses.
It is not intended that pydoc would compete with Sphinx. Sphinx is not core.

pydoc changes
-----------------------------------------------
1. Fix .\ (or ./) problem
Currently, documenting a single module requires the prefix of .\
TODO: Change to check:
    If it's a directory, document all .py files in that directory.
    If it's a Python file, document that.
    If it's one of the special keywords, display that documentation.
    Otherwise, display an error.

Currently:
user@server:/mnt/c/course$ pydoc -w helloFunc.py
no Python documentation found for 'helloFunc.py'

user@server:/mnt/c/course$ pydoc -w ./helloFunc.py
wrote helloFunc.html

TODO: Update the help strings.


2. Fix command line run error message
Currently, passing an argument (that is not a keyword and is not a filename that exists) displays an error message that isn't relevant to this context:

    C:\course> py -3 "C:\Program Files (x86)\Python35-32\Lib\pydoc.py"  notthere.py
    No Python documentation found for 'notthere.py'.
    Use help() to get the interactive help utility.
    Use help(str) for help on the str class.

This changed from Python2:
    C:\course> py -2 "C:\Python27\Lib\pydoc.py"  notthere.py
    No Python documentation found for 'notthere.py'.

TODO: Remove the last two lines of the help output from Py3 pydoc when run from the command line.


3. Add "-h" command-line argument.
TODO: Add "-h This help summary" to the output of pydoc -h
All the existing options are single letter, so make it just -h
Continue the current behaviour of "no command arguments or switches" displays help.


4. TODO: Unknown command line option should give an error.
Currently, pydoc with an unknown command line option doesn't give an error, but just displays the standard help:
    C:\course> py -3 "C:\Program Files (x86)\Python35-32\Lib\pydoc.py" -j
    pydoc - the Python documentation tool

    pydoc <name> ...
        Show text documentation on something.  <name> may be the name of a
        Python keyword, topic, function, module, or package, or a dotte
        ...

This can add confusion. It would be more useful to display a specific error, and refer users to more help, like ls does:
    user@server:~$ ls -j
    ls: invalid option -- 'j'
    Try 'ls --help' for more information.


5. TODO: Improve HTML output
Oh yes. 
There's a few good contributions on this, but I think Frédéric Jolliton's summary is the most succinct.


6. Additional command line argument
To allow certain command line arguments.
Sidenote: Both pod (http://perldoc.perl.org/pod2html.html) and javadoc (http://download.java.net/jdk7u2/docs/technotes/tools/solaris/javadoc.html) have many command line arguments. I'm not proposing for significant changes, but I think the below options would add significantly to the usability, and are modeled on pod and javadoc.
It is not intended that pydoc would compete with Sphinx.
TODO:
+ --help        Same as -h for help
+ --css=        Override the built-in stylesheet
+ --output or -d or --dir   Output directory name
+ --verbose  More verbose output
msg278123 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-05 13:18
You sound like you think we want to get rid of pydoc, which is certainly not true :).  As I understand it it is only the *html* output of pydoc that we are considering deprecating.

Yes, you should open separate issues for your proposed changes.  1 and 2 should be separate issues, 3 and 4 should be combined into a proposal to convert pydoc to using argparse.  5 you can skip, and 6 I don't think will be accepted, although you are welcome to propose it along with some motivating use cases.
msg278126 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-10-05 13:43
I disagree with deprecating HTML output. Pydoc has builtin HTTP server, it provides the documentation with hyperlinks. It would be nice if IDLE use this feature for displaying interactive help.

I think we should enhance HTML output not just by making it looking better, but by adding more interactivity. For example add the ability to collapse classes, methods, etc, add more hyperlinks.
msg278127 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-05 14:07
That sounds like a reasonable argument to me.

To clarify something in my last post: 5 should be skipped in terms of opening new issues because it is already covered by this issue.

If Terry would like to see pydoc enhanced to support idle, then I think that would decide the issue.
msg278166 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-10-06 03:03
Since David added me as nosy, I will address the IDLE issues raised above.

But first, a question.  Does 'deprecating HTML output' merely mean deprecating the old output in favor of new and better HTML output, or deprecating having any html output, which would also mean deprecating the html doc server?  I am strongly against the latter.

Currently, IDLE, in both pyshell and run.py, imports pydoc and sets "pydoc.pager = pydoc.plainpager".  Bypassing pydoc.getpager this way is not documented, but seems perhaps necessary.  Other than this, I believe 'help(xyz)' is treated by IDLE just like any other user code.

Serhiy, msg278126 > "It would be nice if IDLE use this feature"

There are actually two features: hmtl output, and the html server, which uses the default browser to display the output.

Since a year ago, IDLE help displays the html IDLE doc produced by sphinx in a subclass of tkinter Text that feeds the page to a subclass of HTMLParser.  The code is in help.py.  Sphinx's html seems to follow Frédéric's guidelines in msg246917.  I suspect the CSS files are ignored.  We might want to do something similar with pydocs html output.

Raymond: is sphinx's pydoctheme anything like the css you are looking for?  Could it be used as a starting point?
  <link rel="stylesheet" href="../_static/pydoctheme.css" type="text/css" />
  <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />


There have been requests that 'long' help output (for modules and classes) be displayed separately from Shell itself so that it does not obscure history and be easier to refer to.  If we do this, using html instead of plain text would be nice, especially if new features were added.

Serhiy, cont. > "I think we should enhance HTML output not just by making it looking better, but by adding more interactivity. For example add the ability to collapse classes, methods, etc, add more hyperlinks."

I mainly use pydoc server for tkinter.  I would *love* have the repetitive 'inherited methods' section for each class collapsed by default.  Ditto for when I use help interactively.  To me, this is the single worst feature of pydoc output.

David msg278127 > "If Terry would like to see pydoc enhanced to support idle, then I think that would decide the issue."

I am not sure what you mean by 'the issue' and I can't currently think of anything IDLE-specific that pydoc should do, other than be consistent.  However, producing modern, decent looking html output would make html use possible.  Incorporating the current output, as displayed in the server, might be a net negative PR move.


Side note: thinking about how to make a clickable link in a Text widget, I realized that that IDLE already does some syntax tagging for colors (keywords, builtins, def and class identifiers).  The same tags could be bound to double-click or right-click to display help popups, such as
----------------------------------------------------------
The "if" statement is used for conditional execution:

   if_stmt ::= "if" expression ":" suite
               ( "elif" expression ":" suite )*
               ["else" ":" suite]

It selects exactly one of the suites by evaluating the expressions one
by one until one is found to be true (see section Boolean operations
for the definition of true and false); then that suite is executed
(and no other part of the "if" statement is executed or evaluated).
If all expressions are false, the suite of the "else" clause, if
present, is executed.

Related help topics: TRUTHVALUE
--------------------------------------------------------------------
with the grammar chunk highlighted and the related topics clickable.  What IDLE would need here is consistency in the format of the help texts.
msg278191 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-06 13:13
My memory of the thread (I haven't gone back and checked) was that the thought was to drop html output altogether because the sphinx docs were more useful for anyone wanting to use html, and network connectivity was so universal in this day and age that having a local html server for offline uses was of minimal utility.

So your vote that it be kept because it is useful to idle and can be made more useful by improving the html seems to me to decide the issue in favor of making the improvements (given someone willing to do the patch review and any additional needed coding).
msg338700 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2019-03-23 21:27
I've found the HTML to be useful (-w mode, not running a server) for generating quick documentation (much lighter weight commitment than using sphinx).  I show this in my intro classes and the engineers are usually impressed with it.

This is really easy and requires neither configuration files or reST markup:

    $ python -m pydoc -w statistics
    wrote statistics.html

So, my vote it to keep it.

I would still like modern HTML 5 with CSS however.
msg338703 - (view) Author: Andre Roberge (aroberge) * Date: 2019-03-23 21:50
On Sat, Mar 23, 2019 at 6:27 PM Raymond Hettinger <report@bugs.python.org>
wrote:

>
> Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:
>
> I've found the HTML to be useful (-w mode, not running a server) for
> generating quick documentation (much lighter weight commitment than using
> sphinx).  I show this in my intro classes and the engineers are usually
> impressed with it.
>
> This is really easy and requires neither configuration files or reST
> markup:
>
>     $ python -m pydoc -w statistics
>     wrote statistics.html
>
> So, my vote it to keep it.
>
> I would still like modern HTML 5 with CSS however.
>

I submitted code to do this in January 2015 on the bug tracker but, except
for some interest by a single person, my submission was essentially
ignored.  I do not claim it is perfect, but I thought it was a significant
improvement as it did use HTML 5 with CSS.

André Roberge

>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue10716>
> _______________________________________
>
msg338716 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-03-24 05:35
I stumbled upon this issue because I think this is a potential improvement. One of the reasons I don't use it is due to the outdated UI but since it also generates HTML doc for installed packages in a virtual environment I find this to be a useful feature like rdoc (ruby), rustdoc (rust) and are actively maintained helping documentation of third party packages too for local reference.

I have less knowledge about CSS but I tried updating css_v3.diff with master and added boostrap.css from CDN to pydoc which adds spacing along with improving layout and font. Attached is a screenshot that is looks better with the CSS patch and bootstrap. Including bootstrap is not lightweight option in the Python distribution but can be seen as a proof of improvement.
msg338717 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-03-24 05:38
Thanks @aroberge for the patch and your efforts on improving it https://aroberge.blogspot.com/2015/01/scratching-itch-improving-pydoc.html
History
Date User Action Args
2019-03-24 05:38:16xtreaksetmessages: + msg338717
2019-03-24 05:35:58xtreaksetfiles: + Screen Shot 2019-03-24 at 10.58.33 am.png
2019-03-24 05:35:14xtreaksetfiles: + css_master.diff

nosy: + mdk
messages: + msg338716

keywords: + patch
2019-03-23 21:50:25arobergesetmessages: + msg338703
2019-03-23 21:27:42rhettingersetmessages: + msg338700
2019-03-23 19:03:55xtreaksetnosy: + xtreak
2018-01-29 20:52:04rhettingersetversions: + Python 3.8, - Python 3.7
2016-10-06 13:13:17r.david.murraysetmessages: + msg278191
2016-10-06 03:03:31terry.reedysetmessages: + msg278166
2016-10-05 14:07:18r.david.murraysetassignee: eric.araujo ->

messages: + msg278127
nosy: + terry.reedy
2016-10-05 13:43:16serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg278126
2016-10-05 13:18:43r.david.murraysetmessages: + msg278123
2016-10-05 11:15:11PeterLovettsetmessages: + msg278119
2016-10-05 00:36:07r.david.murraysetnosy: + r.david.murray
messages: + msg278098
2016-10-05 00:33:29r.david.murraysetkeywords: - easy
2016-10-04 20:52:36Mariattasetmessages: + msg278090
2016-10-04 20:46:45arobergesetmessages: + msg278089
2016-10-04 19:57:07Mariattasetmessages: + msg278084
2016-10-03 07:10:44berker.peksagsetmessages: + msg277934
versions: + Python 3.7, - Python 3.6
2016-10-02 22:59:15Mariattasetnosy: + Mariatta
2016-08-15 04:27:10PeterLovettsetnosy: + PeterLovett
2015-09-08 18:37:34azsorkinsetnosy: + azsorkin
2015-07-18 23:27:59rhettingersetkeywords: + easy, - patch
versions: + Python 3.6, - Python 3.5
2015-07-18 22:53:03tritiumsetnosy: + tritium
messages: + msg246920
2015-07-18 20:56:15Frédéric Jollitonsetnosy: + Frédéric Jolliton
messages: + msg246917
2015-03-01 13:33:46donmezsetnosy: + donmez
messages: + msg236962
2015-01-29 03:25:51arobergesetmessages: + msg234942
2015-01-29 02:58:09arobergesetmessages: + msg234941
2015-01-29 02:50:57berker.peksagsetversions: + Python 3.5, - Python 3.4
nosy: + berker.peksag

messages: + msg234940

keywords: - gsoc
2015-01-29 02:04:20arobergesetnosy: + aroberge
messages: + msg234936
2013-03-02 12:55:04ezio.melottisetmessages: + msg183319
2013-03-02 00:00:54ron_adamsetmessages: + msg183304
2012-07-13 19:12:48ezio.melottisetmessages: + msg165406
2012-07-13 19:00:50vterronsetnosy: + vterron
messages: + msg165401
2012-07-05 18:36:48eric.araujosetassignee: lukasz.langa -> eric.araujo
stage: patch review
versions: + Python 3.4, - Python 3.3
2012-07-05 18:36:36eric.araujosettitle: Modernize pydoc to use CSS -> Modernize pydoc to use better HTML and separate CSS
2011-03-11 22:14:57ezio.melottisetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg130629
2011-01-30 16:12:36ron_adamsetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg127542
2011-01-02 23:15:31ron_adamsetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg125122
2011-01-02 22:40:38georg.brandlsetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg125113
2011-01-02 22:19:52ron_adamsetfiles: + css_v3.diff
nosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg125111
2011-01-02 13:54:59ezio.melottisetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg125049
2011-01-02 07:43:06ron_adamsetfiles: - css_v1.diff
nosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
2011-01-02 07:42:47ron_adamsetfiles: - pydoc sample html files.zip
nosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
2011-01-02 07:42:40ron_adamsetfiles: - defaultstyle.css
nosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
2011-01-02 07:42:23ron_adamsetfiles: + css_v2.diff
nosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg125033
2010-12-29 18:23:09ron_adamsetnosy: georg.brandl, rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg124863
2010-12-29 17:14:33georg.brandlsetnosy: + georg.brandl
messages: + msg124858
2010-12-29 17:12:16ron_adamsetnosy: rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg124857
2010-12-28 22:48:57ron_adamsetfiles: + css_v1.diff

messages: + msg124825
keywords: + patch
nosy: rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
2010-12-18 18:43:21rhettingersetnosy: rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg124306
2010-12-17 11:47:02eric.araujosetnosy: rhettinger, belopolsky, ron_adam, ezio.melotti, eric.araujo, lukasz.langa
messages: + msg124195
2010-12-17 03:42:55ezio.melottisetnosy: + ezio.melotti
messages: + msg124181
2010-12-17 03:41:40ron_adamsetfiles: + pydoc sample html files.zip
nosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124179
2010-12-17 03:32:20rhettingersetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124177
2010-12-17 03:06:48ron_adamsetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124176
2010-12-16 23:34:00rhettingersetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124164
2010-12-16 22:40:21ron_adamsetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124162
2010-12-16 21:42:20rhettingersetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124159
2010-12-16 21:17:43eric.araujosetnosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124158
2010-12-16 21:12:52ron_adamsetfiles: + defaultstyle.css
nosy: rhettinger, belopolsky, ron_adam, eric.araujo, lukasz.langa
messages: + msg124157
2010-12-16 19:23:00eric.araujosetnosy: + eric.araujo
2010-12-16 18:22:23belopolskysetnosy: rhettinger, belopolsky, ron_adam, lukasz.langa
messages: + msg124147
2010-12-16 18:20:04belopolskysetnosy: + belopolsky, ron_adam
messages: + msg124144
2010-12-16 10:28:21lukasz.langasetassignee: lukasz.langa

nosy: + lukasz.langa
2010-12-16 09:25:37rhettingercreate