Defeat many warnings and related fixes building documentation

* Mask various CLI options with exclamation marks (!)
* Correct link to CLI section in zipapp
* Make CLI section label in timeit less ambiguous

The following warnings from unittest remain. Ideally they should be linked to the Python CLI documentation, but this does not seem possible because the “unittest” documentation defines its own CLI options.

Doc/library/unittest.rst:1965: WARNING: unknown option: -Wd
Doc/library/unittest.rst:1965: WARNING: unknown option: -Wa
Doc/library/unittest.rst:2051: WARNING: unknown option: -W

FTR the warnings that I am fixing were apparently enabled in Sphinx 1.3.4, reverted in 1.3.6, and added in 1.4.

> FTR the warnings that I am fixing were apparently enabled in Sphinx 1.3.4, reverted in 1.3.6, and added in 1.4.

Would it be possible to turn off the warning?

Doc/using/cmdline.py uses ".. cmdoption::". If we cannot turn off the warning on option, maybe we should use a new :cmdoption:`xxx` which wouldn't emit a warning?

I'm not strongly opposed to doc-warnings.patch.

I understand that Sphinx expects the exact option, for example -W, whereas the doc uses option with parameter like -Wd. Sphinx is unable to link to the option.

Maybe we need something like :cmdoption:`-Wd <-W>`: display -Wd but link to -W?

I’m not an expert on Sphinx. Maybe <http://www.sphinx-doc.org/en/stable/config.html#confval-suppress_warnings> could help with disabling these warnings. Considering there are so many false positives, it might be best to disable the warning. Though my patch does have a few genuine improvements as well.

I already used the :option:`-Wd <-W>` syntax in some cases; see Doc/library/warnings.rst for example. The problem in unittest.rst is I think Sphinx is trying to find a -W option defined within unittest.rst (with the unittest --buffer etc options), rather than the main Python options from using/cmdline.rst.

New changeset 23a884c32a39 by Martin Panter in branch '3.5':
Issue #26638: Fix links to some CLI options and section headings
https://hg.python.org/cpython/rev/23a884c32a39

New changeset 68b84643dffd by Martin Panter in branch 'default':
Issue #26638: Merge link fixes from 3.5
https://hg.python.org/cpython/rev/68b84643dffd

New changeset 3e0321584f23 by Martin Panter in branch '2.7':
Issue #26638: Fix links to some CLI options
https://hg.python.org/cpython/rev/3e0321584f23

I committed the parts of my patch that improve real links in the end result.

doc-warnings.v2.patch has the remaining changes which add exclamation marks to mask undefined options.

This patch disables the warning, but I tend to think it is not a good long-term solution.

New changeset 3996dd17ca21 by Martin Panter in branch '3.5':
Issue #26638: Don’t link 2to3 -l option to the tarfile -l option
https://hg.python.org/cpython/rev/3996dd17ca21

New changeset d7cbaaba59e8 by Martin Panter in branch 'default':
Issue #26638: Merge CLI link fix from 3.5
https://hg.python.org/cpython/rev/d7cbaaba59e8

I built the 3.5 docs on Win10 with Sphinx 1.3.3 and corresponding dependencies and got a new warning as a result of 8ba0000c90be for #26462:
F:\Python\dev\36\Doc\whatsnew\3.2.rst:2063: WARNING: Pygments lexer name 'doscon' is not known

Thinking the problem was an out-of-date pygments module, I updated sphinx and dependencies (to 1.4.6, etc).  I was right that updating pygments fixed the warning above, but now I get 140 warnings!!!

4 are from the reading phase for configparser.rst.  The first:

F:\Python\dev\35\Doc\library\configparser.rst:240: WARNING: Could not lex literal_block as "ini". Highlighting skipped.

I presume this could be fixed, but I don't know how.

136 are about "<path:  WARNING: unknown option:  -x."

Sphinx, at least with how we use it, and the Python docs are out of sync here. I think we need more eyes on this.

"This patch disables the warning," ??? Not here.

" but I tend to think it is not a good long-term solution."

Printing 136 warnings is also not a good solution.  I rebuilt the docs locally because I was planning to patch a misspelling I found while reading the doc.  Looking in a list of several pages for a warning I might have caused is failure-prone and no fun.  In the Windows console, the page-up and page-down keys do not page up and down.  To not miss anything, I would have to carefully use a mouse wheel or scrollbar clicks.

I am building with "sphinx-build -bhtml . build/html" is a .bat file.  Is there anything I can add to suppress only 'unknown option' warnings?

Terry: for now, you can downgrade Sphinx to 1.3.6 as it is known to build with no warnings (that's the version the Docs buildbot currently uses).

Only 138 warnings with 2.7.

Zack, how do I downgrade.  I examined the result of pip install -h but cannot see how.  I presume that 'sphinx' and '1.3.6' should be combined into a '<requirement specifier>' but this latter is not defined and the things I tried, including the output format of 'pip list' did not work.

Does anything else need to be downgraded?

The syntax is `<pip> install "<package><comparison><version>"`, so in this case 'pip install "Sphinx==1.3.6"` (or `pip install "Sphinx<1.4"`) should do it for you.  That ought to also take care of downgrading any other packages that need it (though I don't think there are any), or at least tell you which ones should be downgraded and to what.

I'm not sure why there are both `pip list` and `pip freeze`, but `pip freeze` is the one that outputs usable syntax for a requirement specifier.

Thank you, that worked -- and should have been documented in the help output.  All the sphinx dependencies were inequalities, and rebuilding worked.

The devguide only says "You are expected to have installed a recent version of Sphinx on your system or in a virtualenv,".  Could we add

"We currently recommend 'python -m pip install sphinx==1.3.6' as the easiest way to install and the best version for most people to install.  See https://bugs.python.org/issue26638 for why."

(I am assuming that pip works on all systems, not just Windows.)  The sentence finishes with "so that the Makefile can find the sphinx-build command."  On Windows. this is most easily done by installing Sphinx with a Python version that was added to PATH when installed.

Terry: the four "ini" syntax highlighting warnings were also mentioned in <https://bugs.python.org/issue26462#msg271241>. In that bug, we fixed many other similar warnings, but I think we left those as being less easy to fix.

One thing (other than suppress_warnings = ['ref.option']) that might help you see warnings relevant to your change is if you first build the docs without your change, ignore all those warnings, and then rebuild with your change. The second build may produce less irrelevant warnings.

Patch  (v3) is fine, except it no longer applies cleanly. I have a patch against the tip which I could upload if needed.

I would also be in favour of disabling the unittest options directly, currently no link is generated and it doesn't seem like there is a straightforward solution in sight.

New changeset 307d7568b6ae by Martin Panter in branch '3.5':
Issue #26638: Mask undefined CLI options to defeat new Sphinx warnings
https://hg.python.org/cpython/rev/307d7568b6ae

New changeset 57f4ba6b29bf by Martin Panter in branch '3.5':
Issue #26638: Work around more CLI options that can’t be linked
https://hg.python.org/cpython/rev/57f4ba6b29bf

New changeset a0d272fbc7de by Martin Panter in branch '3.6':
Issue #26638: Merge option warning fixes from 3.5 into 3.6
https://hg.python.org/cpython/rev/a0d272fbc7de

New changeset 85e2cfe5b12d by Martin Panter in branch 'default':
Issue #26638: Merge option warning fixes from 3.6
https://hg.python.org/cpython/rev/85e2cfe5b12d

New changeset c4b934a77a08 by Martin Panter in branch '2.7':
Issue #26638: Disable inappropriate links to Python interpreter options
https://hg.python.org/cpython/rev/c4b934a77a08

New changeset 0ff00d53d6a9 by Martin Panter in branch '2.7':
Issue #26638: Mask undefined CLI options to defeat new Sphinx warnings
https://hg.python.org/cpython/rev/0ff00d53d6a9

New changeset cf91d48aa353 by Martin Panter in branch '2.7':
Issue #26638: Cannot directly link to main option from the “timeit” module
https://hg.python.org/cpython/rev/cf91d48aa353

For the “unittest” module, I added separate text with a link: (see Warning control). Similarly for “timeit” in 2.7. Now the only Sphinx warnings I get are a handful about syntax highlighting, and two genuine ones relating to the missing -b documentation (Issue 11681).