This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: Doctest directives and comments missing from code samples
Type: behavior Stage:
Components: Documentation Versions: Python 3.8, Python 3.7, Python 2.7
Status: open Resolution: later
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Anthony Sottile, JDLH, cheryl.sabella, docs@python, eric.araujo, ezio.melotti, gregory.p.smith, jdemeyer, mdk, minghua, miss-islington, steven.daprano, terry.reedy, willingc
Priority: normal Keywords: patch

Created on 2019-04-20 10:43 by steven.daprano, last changed 2022-04-11 14:59 by admin.

File name Uploaded Description Edit
missing_directives.png steven.daprano, 2019-04-20 10:43
Pull Requests
URL Status Linked Edit
PR 13125 closed python-dev, 2019-05-06 18:32
PR 16024 merged mdk, 2019-09-12 10:53
PR 16030 merged miss-islington, 2019-09-12 11:24
PR 23620 merged mdk, 2020-12-02 23:10
Messages (17)
msg340570 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2019-04-20 10:43
(Apologies if this is the wrong place for reporting website bugs.)

The website is not rendering doctest directives or comments, either that or the comments have been stripped from the examples.

On the doctest page itself, all the comments are missing:

The first example says:

    >>> print(list(range(20))) 
    [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
    10,  11, 12, 13, 14, 15, 16, 17, 18, 19]

but without the directive, the test would fail. Screen shot attached.

Doctest directives are also missing from here:

My browser: Firefox 45.1.1

Also checked with text browser "lynx".
msg340959 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-04-27 02:20
I verified that the line in Doc/library/doctest.rst has the comment.
For example, this test passes::

   >>> print(list(range(20))) # doctest: +NORMALIZE_WHITESPACE
   [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
   10,  11, 12, 13, 14, 15, 16, 17, 18, 19]
However, the comment is omitted from the .html built on Windows by Sphinx 1.8.1.
<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">)))</span> 
   ***html for comment should be here***
<span class="go">[0,   1,  2,  3,  4,  5,  6,  7,  8,  9,</span>
<span class="go">10,  11, 12, 13, 14, 15, 16, 17, 18, 19]</span>
To me, this is a bug with building the .html for doctest.rst.  Comments are preserved elsewhere.  For instance, the code example for  The .rst file has
   The resulting list is sorted alphabetically.  For example:

      >>> import struct
      >>> dir()   # show the names in the module namespace  # doctest: +SKIP
Both use 3-space colon + indents to mean 'example block'.  The only difference is '::' versus ':'. says :: is required.  idle.rst also has a code example with comments displayed (I just submitted a PR to not suppress color highlighting.)

I leave it to the doc experts to discover why the comments are not included in doctest.html.
msg342648 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-16 14:45
Isn't it a *feature* that those doctest directives are not shown? Those directives are meant for the doctest module only, not for the reader of the rendered documentation.
msg342651 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2019-05-16 15:05
OP is about the documentation page for doctest itself!
msg342652 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2019-05-16 15:25
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.  The patch only exposes them for doctest.html and not for ctypes or anywhere else.

They really should not be in the dir example code that I linked to.
The problem there are the double comments with both a real comment and a directive. does not say anything about '::' causing suppression of comments and ':' leaving them.  It is misleading in implying the '::' is required for a code block.
msg342655 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2019-05-16 15:44
> Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.

Thanks for clarifying. I missed that.
msg342664 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2019-05-16 17:38
> Doctest directives in code examples should be suppressed everywhere 
> *except* in the doctest.html examples showing how to use directives.  
> The patch only exposes them for doctest.html and not for ctypes or 
> anywhere else.

Thanks for the patch, and the extra information, but I disagree with the 
decision to suppress the directives.

The reason I found this problem in the first case was that I started 
with the ctypes documentation, where it says:

"Since some code samples behave differently under Linux, Windows, or Mac 
OS X, they contain doctest directives in comments."

and I was very keen to see those directives so I could learn the right 
way to deal with platform-dependent doctests. I was very confused that 
they weren't visible.

I think that doctest directives are as much a part of documenting 
correct usage as any other part of the example code, and they are 
(semi-)human readable and (almost) self-documenting.

Consider this example from ctypes:

    >>> c_wchar_p("Hello, World")

In the absence of a directive, but knowing that it may have been 
surpressed, I don't know how to interpret that. Is the output some 
arbitrarily chosen value that doctest ought to skip? Or is that the 
actual output that c_wchar_p("Hello, World") will return every single 
time without fail?

If I was a ctypes expert, it might be blindingly obvious to me, but I'm 
not, so I'm left in the dark. I don't know whether I should expect that 
precise output each and every time, or something platform and 
implementation specific.

If the directive #doctest:+SKIP was visible, I would know that it was an 
arbitrarily chosen example. 

My preference would be:

- keep the doctest directives visible, everywhere;

- make them a clickable link to the appropriate section 
  in the doctest documentation;

- and, if possible, on mouse-over, they should display a
  tooltip with a message like 

  "The output of this example is arbitrary."

Or similar wording.

> They really should not be in the dir example code that I linked to.

On the contrary: I think that the presence of the +SKIP directive helps 
demonstrate that the output shown is a made-up example, not normative.

(Of course it helps that I know doctest, but even if I didn't, the 
tooltip message would help.)
msg344714 - (view) Author: Cheryl Sabella (cheryl.sabella) * (Python committer) Date: 2019-06-05 12:42
;tldr;   There is a global configuration flag to show all the doctest directives for all the docs that are built.  The default is to suppress the doctest directives.  If a global setting isn't desired, then each doctest example will have to be changed individually.

I've looked at the Sphinx documentation for this and there are a few points to reference. 

* `::` defines a Literal block in reST. [1]  A literal block doesn't do anything special except from the last paragraph is this section of the docs: 
   "Code highlighting can be enabled for these literal blocks on a document-wide basis using the `highlight` directive and on a project-wide basis using the `highlight_language` configuration option. The code-block directive can be used to set highlighting on a block-by-block basis. These directives are discussed later."  CPython has the `highlight-language` set to python3.  

* A doctest block does not need to be in a Literal Block. [2]  They are identified automatically. [3]  According to the doc, doctest directives are suppressed from presentation:

    Also, you can give inline doctest options, like in doctest:

    >>>   # doctest: +SKIP, 1, 1)

    They will be respected when the test is run, but stripped from presentation output.

* Any section (doesn't need the single colon (`:`) can have the `code-block` directive on it. [4]  The `code-block` directive can define the language for highlighting.  

Note that in the Sphinx doctest [4] documentation, there is a config option `trim_doctest_flags`, which is True by default.  Setting this to False will show all the doctest directives for all the doctests in the documentation.

The nice thing about the doc build recognizing a section as a doctest is that it adds the toggle in the upper right of the block to 'Hide the toggle and output' so that the example can be copy and pasted more easily.  Changing it into something other than a doctest (like using a `code-block` directive, takes away this option.  Using `trim_doctest_flags=False` retains the hide button.

1.  Change global setting to false to show all doctest directives.
2.  Change only the blocks where the doctest directives need to be shown, but this makes them lose the 'Hide prompt and output' button.
3.  Add a new option under 'code-block' to control the displaying of the doctest directives at the code-block level. This would override the global setting.  I think this option could be added locally, but may need to be done upstream in Sphinx.

msg352093 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2019-09-12 10:28
I opened an issue on the sphinx-doc repo [1] to check if it would be possible to have an option in doctest blocks to not trim them.

We previously had a hack in Doc/tools/extensions/ to patch sphinx to not trim them for the doctest.rst file. But sphinx deprecated the hack we used :(

msg352120 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-09-12 11:23
New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith (Julien Palard) in branch 'master':
bpo-36675: Remove obsolete code. (GH-16024)
msg352128 - (view) Author: miss-islington (miss-islington) Date: 2019-09-12 11:31
New changeset 94a684734f669eab02b5c915394b749ccf936449 by Miss Islington (bot) in branch '3.8':
bpo-36675: Remove obsolete code. (GH-16024)
msg367173 - (view) Author: Jim DeLaHunt (JDLH) * Date: 2020-04-24 07:29
We discovered and fixed this same problem back in 2011-2012 with #12947 . 

That was apparently the source of the monkeypatch that was removed as "obselete code" on 2019-09-12. That old issue commentary has some suggestions about other workarounds. This includes adding explanatory text about the fact that doctest directives are missing from the examples which should be showing them. Maybe some of those workarounds would be effective for us now.
msg383075 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2020-12-15 16:23
New changeset c8a10d2fabff492ab352501c316baf5f2fc6510e by Julien Palard in branch 'master':
bpo-36675: Doc: Reveal doctest directives (GH-23620)
msg383732 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2020-12-25 10:32
Happy Christmas, everybody involved in this issue! I'm happy to announce this issue is resolved since a few days \o/
msg384361 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2021-01-05 00:42
should the minimum sphinx version be bumped and/or this reverted:

this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04

The directive used here requires sphinx>=3.2.0

I notice some other attempts have been made to make the docs more compatible with sphinx 1.x in this release as well so there might be conflicting directions here:
msg385632 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2021-01-25 14:49
Due to this is sadly un-fixed.

Either we find another way to fix this, either we wait 3 releases and we re-commit
msg385661 - (view) Author: Jim DeLaHunt (JDLH) * Date: 2021-01-25 20:57
My goodness, things get complex sometimes. 

If we cannot make Sphinx preserve doctest directives and comments, perhaps we should go back to the historical bug discussion to look at workarounds which we considered earlier. For instance, maybe we should modify the text surrounding the examples to explain that doctests directives should appear there, but that our tool chain currently removes them.

At the moment, I see doctest directives in the doctest source code at: 
which do not appear in the corresponding HTML output at: 

How about rewording the text before each of those examples? Or maybe finding some way to show those examples as literal text which Sphinx won't modify, even if it is not formatted like Python code examples?

By the way, the discussion of this same problem back in 2011-2012 is at #12947 . At that time, we applied a "monkey patch" to the Sphinx code. I haven't read the discussion closely enough to figure out if such a patch would help now.
Date User Action Args
2022-04-11 14:59:14adminsetgithub: 80856
2021-10-26 04:00:41minghuasetnosy: + minghua
2021-01-25 20:57:41JDLHsetmessages: + msg385661
2021-01-25 14:49:03mdksetstatus: closed -> open
resolution: fixed -> later
messages: + msg385632

stage: resolved ->
2021-01-05 00:42:35Anthony Sottilesetnosy: + Anthony Sottile
messages: + msg384361
2020-12-25 10:32:51mdksetstatus: open -> closed
resolution: fixed
messages: + msg383732

stage: patch review -> resolved
2020-12-15 16:23:10mdksetmessages: + msg383075
2020-12-02 23:15:42mdklinkissue42324 superseder
2020-12-02 23:10:28mdksetpull_requests: + pull_request22488
2020-04-24 07:29:20JDLHsetnosy: + JDLH
messages: + msg367173
2019-09-12 11:31:20miss-islingtonsetnosy: + miss-islington
messages: + msg352128
2019-09-12 11:24:07miss-islingtonsetpull_requests: + pull_request15653
2019-09-12 11:23:57gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg352120
2019-09-12 10:53:25mdksetpull_requests: + pull_request15647
2019-09-12 10:28:39mdksetmessages: + msg352093
2019-06-05 12:42:33cheryl.sabellasetnosy: + cheryl.sabella
messages: + msg344714
2019-05-16 17:38:34steven.dapranosetmessages: + msg342664
2019-05-16 15:44:03jdemeyersetmessages: + msg342655
2019-05-16 15:25:40terry.reedysetmessages: + msg342652
2019-05-16 15:05:24eric.araujosetmessages: + msg342651
2019-05-16 14:45:36jdemeyersetnosy: + jdemeyer
messages: + msg342648
2019-05-06 18:32:21python-devsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request13038
2019-04-27 02:20:55terry.reedysetnosy: + eric.araujo, willingc, terry.reedy, ezio.melotti
title: Doctest directives and comments not visible or missing from code samples -> Doctest directives and comments missing from code samples
messages: + msg340959

versions: + Python 2.7, Python 3.7, Python 3.8
stage: needs patch
2019-04-20 10:58:36xtreaksetnosy: + mdk
2019-04-20 10:43:50steven.dapranocreate