(Apologies if this is the wrong place for reporting website bugs.)

The website is not rendering doctest directives or comments, either that or the comments have been stripped from the examples.

On the doctest page itself, all the comments are missing:

https://docs.python.org/3/library/doctest.html#directives

The first example says:

    >>> print(list(range(20))) 
    [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
    10,  11, 12, 13, 14, 15, 16, 17, 18, 19]

but without the directive, the test would fail. Screen shot attached.

Doctest directives are also missing from here:

https://docs.python.org/3/library/ctypes.html



My browser: Firefox 45.1.1

Also checked with text browser "lynx".

I verified that the line in Doc/library/doctest.rst has the comment.
"
For example, this test passes::

   >>> print(list(range(20))) # doctest: +NORMALIZE_WHITESPACE
   [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
   10,  11, 12, 13, 14, 15, 16, 17, 18, 19]
"
However, the comment is omitted from the .html built on Windows by Sphinx 1.8.1.
"
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">)))</span> 
   ***html for comment should be here***
<span class="go">[0,   1,  2,  3,  4,  5,  6,  7,  8,  9,</span>
<span class="go">10,  11, 12, 13, 14, 15, 16, 17, 18, 19]</span>
</pre></div>
</div>
"
To me, this is a bug with building the .html for doctest.rst.  Comments are preserved elsewhere.  For instance, the code example for https://docs.python.org/3/library/functions.html#dir.  The .rst file has
"
   The resulting list is sorted alphabetically.  For example:

      >>> import struct
      >>> dir()   # show the names in the module namespace  # doctest: +SKIP
"
Both use 3-space colon + indents to mean 'example block'.  The only difference is '::' versus ':'.  https://devguide.python.org/documenting/#source-code says :: is required.  idle.rst also has a code example with comments displayed (I just submitted a PR to not suppress color highlighting.)

I leave it to the doc experts to discover why the comments are not included in doctest.html.

Isn't it a *feature* that those doctest directives are not shown? Those directives are meant for the doctest module only, not for the reader of the rendered documentation.

OP is about the documentation page for doctest itself!

Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.  The patch only exposes them for doctest.html and not for ctypes or anywhere else.

They really should not be in the dir example code that I linked to.
https://docs.python.org/3/library/functions.html#dir
The problem there are the double comments with both a real comment and a directive.

https://devguide.python.org/documenting/#source-code does not say anything about '::' causing suppression of comments and ':' leaving them.  It is misleading in implying the '::' is required for a code block.

> Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.

Thanks for clarifying. I missed that.

> Doctest directives in code examples should be suppressed everywhere 
> *except* in the doctest.html examples showing how to use directives.  
> The patch only exposes them for doctest.html and not for ctypes or 
> anywhere else.

Thanks for the patch, and the extra information, but I disagree with the 
decision to suppress the directives.

The reason I found this problem in the first case was that I started 
with the ctypes documentation, where it says:

"Since some code samples behave differently under Linux, Windows, or Mac 
OS X, they contain doctest directives in comments."

and I was very keen to see those directives so I could learn the right 
way to deal with platform-dependent doctests. I was very confused that 
they weren't visible.

https://docs.python.org/3/library/ctypes.html

I think that doctest directives are as much a part of documenting 
correct usage as any other part of the example code, and they are 
(semi-)human readable and (almost) self-documenting.

Consider this example from ctypes:

    >>> c_wchar_p("Hello, World")
    c_wchar_p(140018365411392)

In the absence of a directive, but knowing that it may have been 
surpressed, I don't know how to interpret that. Is the output some 
arbitrarily chosen value that doctest ought to skip? Or is that the 
actual output that c_wchar_p("Hello, World") will return every single 
time without fail?

If I was a ctypes expert, it might be blindingly obvious to me, but I'm 
not, so I'm left in the dark. I don't know whether I should expect that 
precise output each and every time, or something platform and 
implementation specific.

If the directive #doctest:+SKIP was visible, I would know that it was an 
arbitrarily chosen example. 

My preference would be:

- keep the doctest directives visible, everywhere;

- make them a clickable link to the appropriate section 
  in the doctest documentation;

- and, if possible, on mouse-over, they should display a
  tooltip with a message like 

  "The output of this example is arbitrary."

Or similar wording.

> They really should not be in the dir example code that I linked to.
> https://docs.python.org/3/library/functions.html#dir

On the contrary: I think that the presence of the +SKIP directive helps 
demonstrate that the output shown is a made-up example, not normative.

(Of course it helps that I know doctest, but even if I didn't, the 
tooltip message would help.)

;tldr;   There is a global configuration flag to show all the doctest directives for all the docs that are built.  The default is to suppress the doctest directives.  If a global setting isn't desired, then each doctest example will have to be changed individually.


I've looked at the Sphinx documentation for this and there are a few points to reference. 

* `::` defines a Literal block in reST. [1]  A literal block doesn't do anything special except from the last paragraph is this section of the docs: 
   "Code highlighting can be enabled for these literal blocks on a document-wide basis using the `highlight` directive and on a project-wide basis using the `highlight_language` configuration option. The code-block directive can be used to set highlighting on a block-by-block basis. These directives are discussed later."  CPython has the `highlight-language` set to python3.  

* A doctest block does not need to be in a Literal Block. [2]  They are identified automatically. [3]  According to the doc, doctest directives are suppressed from presentation:

    Also, you can give inline doctest options, like in doctest:

    >>> datetime.date.now()   # doctest: +SKIP
    datetime.date(2008, 1, 1)

    They will be respected when the test is run, but stripped from presentation output.

* Any section (doesn't need the single colon (`:`) can have the `code-block` directive on it. [4]  The `code-block` directive can define the language for highlighting.  

Note that in the Sphinx doctest [4] documentation, there is a config option `trim_doctest_flags`, which is True by default.  Setting this to False will show all the doctest directives for all the doctests in the documentation.

The nice thing about the doc build recognizing a section as a doctest is that it adds the toggle in the upper right of the block to 'Hide the toggle and output' so that the example can be copy and pasted more easily.  Changing it into something other than a doctest (like using a `code-block` directive, takes away this option.  Using `trim_doctest_flags=False` retains the hide button.

Options:
1.  Change global setting to false to show all doctest directives.
2.  Change only the blocks where the doctest directives need to be shown, but this makes them lose the 'Hide prompt and output' button.
3.  Add a new option under 'code-block' to control the displaying of the doctest directives at the code-block level. This would override the global setting.  I think this option could be added locally, but may need to be done upstream in Sphinx.



[1] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#literal-blocks
[2] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#doctest-blocks
[3] http://www.sphinx-doc.org/en/stable/usage/extensions/doctest.html
[4] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/directives.html#showing-code-examples

I opened an issue on the sphinx-doc repo [1] to check if it would be possible to have an option in doctest blocks to not trim them.

We previously had a hack in Doc/tools/extensions/pyspecific.py to patch sphinx to not trim them for the doctest.rst file. But sphinx deprecated the hack we used :(

[1]: https://github.com/sphinx-doc/sphinx/issues/6698

New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith (Julien Palard) in branch 'master':
bpo-36675: Remove obsolete code. (GH-16024)
https://github.com/python/cpython/commit/2c910c1e732c9a3ec4c67a7c43d789d6c729304a

New changeset 94a684734f669eab02b5c915394b749ccf936449 by Miss Islington (bot) in branch '3.8':
bpo-36675: Remove obsolete code. (GH-16024)
https://github.com/python/cpython/commit/94a684734f669eab02b5c915394b749ccf936449

We discovered and fixed this same problem back in 2011-2012 with #12947 . 

That was apparently the source of the monkeypatch that was removed as "obselete code" on 2019-09-12. That old issue commentary has some suggestions about other workarounds. This includes adding explanatory text about the fact that doctest directives are missing from the examples which should be showing them. Maybe some of those workarounds would be effective for us now.

New changeset c8a10d2fabff492ab352501c316baf5f2fc6510e by Julien Palard in branch 'master':
bpo-36675: Doc: Reveal doctest directives (GH-23620)
https://github.com/python/cpython/commit/c8a10d2fabff492ab352501c316baf5f2fc6510e

Happy Christmas, everybody involved in this issue! I'm happy to announce this issue is resolved since a few days \o/

should the minimum sphinx version be bumped and/or this reverted: https://github.com/python/cpython/blob/f7f0ed59bcc41ed20674d4b2aa443d3b79e725f4/Doc/conf.py#L48

this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04

The directive used here requires sphinx>=3.2.0

I notice some other attempts have been made to make the docs more compatible with sphinx 1.x in this release as well so there might be conflicting directions here: https://github.com/python/cpython/commit/b63a620014b67a6e63d10783149c41baaf59def8

Due to https://github.com/python/cpython/pull/24282 this is sadly un-fixed.

Either we find another way to fix this, either we wait 3 releases and we re-commit https://github.com/python/cpython/pull/23620.

My goodness, things get complex sometimes. 

If we cannot make Sphinx preserve doctest directives and comments, perhaps we should go back to the historical bug discussion to look at workarounds which we considered earlier. For instance, maybe we should modify the text surrounding the examples to explain that doctests directives should appear there, but that our tool chain currently removes them.

At the moment, I see doctest directives in the doctest source code at: 
https://github.com/python/cpython/blob/master/Doc/library/doctest.rst#directives 
which do not appear in the corresponding HTML output at:
https://docs.python.org/3/library/doctest.html#directives 

How about rewording the text before each of those examples? Or maybe finding some way to show those examples as literal text which Sphinx won't modify, even if it is not formatted like Python code examples?

By the way, the discussion of this same problem back in 2011-2012 is at #12947 . At that time, we applied a "monkey patch" to the Sphinx code. I haven't read the discussion closely enough to figure out if such a patch would help now.