classification
Title: Fix doctest runable examples in python manual
Type: Stage: patch review
Components: Documentation Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: open Resolution: accepted
Dependencies: 10224 Superseder:
Assigned To: Nosy List: andymaier, belopolsky, docs@python, eric.araujo, ezio.melotti, georg.brandl, lukasz.langa, rhettinger, terry.reedy
Priority: normal Keywords: patch

Created on 2010-10-29 02:18 by belopolsky, last changed 2016-07-16 01:27 by belopolsky.

Files
File name Uploaded Description Edit
issue10225-r27.diff belopolsky, 2010-10-29 07:49 Patch agains 2.7 branch
issue10225a-r27.diff belopolsky, 2010-10-29 15:40 Patch against 2.7 branch
sorting-howto.diff belopolsky, 2011-01-12 04:32 review
issue10225-py3k.diff belopolsky, 2011-01-12 16:42 review
Messages (21)
msg119840 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-29 02:18
As noted in issue 10224, python 3.x documentation is not being built with python 3.x yet, so a simple cd Doc; make doctest does not work.  However, hg trunk version os sphinx works with python 3.x and can be used to validate examples in ReST documentation.
msg119841 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-10-29 02:20
> hg trunk version [of] sphinx works with python 3.x and can be used to
> validate examples in ReST documentation.

How do you replace “make doctest”?
msg119842 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-29 02:22
On Thu, Oct 28, 2010 at 10:20 PM, Éric Araujo <report@bugs.python.org> wrote:

>
> How do you replace “make doctest”?

$ sphinx-build -b html -d Doc/build3/doctrees -D latex_paper_size=
Doc  Doc/build3/html

but I had to run 2to3 on Doc/conf.py and Doc/tools first.
msg119869 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-29 07:19
I started with 2.7 branch because some of the issues are the same there, but the tools work better at the moment.  I a posting a work-in-progress patch to solicit early feedback.
msg119908 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-29 15:40
I am attaching a new patch which fixes all but two doctest examples.

I suspect that the remaining failures are due to a bug in sphinx. (The examples are executed even though marked up with ::).
msg119917 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-10-29 16:28
I started porting the patch to 3.x and it looks like I've uncovered another bug in sphinx:

$ sphinx-build -b doctest -d build/doctrees  . build/doctest library/traceback.rst
..
**********************************************************************
File "library/traceback.rst", line 327, in default
..

Differences (ndiff with -expected +actual):
      *** print_tb:
    -   File "<doctest...>", line 10, in <module>
    ?                 ^^^
    +   File "<doctest default[0]>", line 10, in <module>
    ?                 ^^^^^^^^^^^
          lumberjack()

(I added +REPORT_NDIFF to testoutput options for clarity.)

I even tried adding +ELLIPSIS to options, but this did not help.  The same example works in 2.7.
msg125860 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-01-09 20:52
As I understand this, your patch would fix numerous errors in the docs, but has not been applied. It would be nice to get the fixes into 3.2. What sort of feedback do you want? Proofreading of text? Recheck of doctest?

Does sphinx suppress doctest comments like
     #doctest: +IGNORE_EXCEPTION_DETAIL
?

doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False) fails 23 of 37 attempts, so improvement is needed ;-).
msg125906 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-10 17:37
On Sun, Jan 9, 2011 at 3:52 PM, Terry J. Reedy <report@bugs.python.org> wrote:
> .. It would be nice to get the fixes into 3.2.

Yes, it would be nice, but I don't have a 3.2 patch (yet).  As you can
see from my last comment, I ran into a bug in the alpha version of
sphinx that I used to validate py3k docs.

> What sort of feedback do you want? Proofreading of text? Recheck of doctest?
>

All of the above.  Some of the fixes may be at the expense of
readability.   I would appreciate feedback in the instances when human
and computer reader's interests are in conflict.   Some examples are
"fixed" by excluding them from being checked.  Better ideas are
welcome.

> Does sphinx suppress doctest comments like
>     #doctest: +IGNORE_EXCEPTION_DETAIL
> ?

Yes.

>
> doctest.testfile("C:/programs/PyDev/py32/Doc/howto/sorting.rst", module_relative = False)
> fails 23 of 37 attempts, so improvement is needed ;-).

You cannot run documentation examples with a plain doctest.   You have
to use sphinx-build.
msg126050 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-01-11 23:10
Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed. See #10875 where is was very helpful. So your last comment puzzles me.

In any case, your patch is too big to digest at once. I extracted the part for howto/sorting.rst and tried to apply to 3.2, but the file has changed too much (at least for TortoiseSVN). So I pasted the '... 's (and adjusted spacing) where needed, and did the two other changes you had. This left just two failures -- for the 2.7 code using 'cmp=xxx' args. With '# doctest: +SKIP' added twice, doctest passes. Patch commited for 3.2 as r87946.

> instances when human and computer reader's interests are in conflict.
What is one (or more) that you were thinking of?
msg126051 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-11 23:17
On Tue, Jan 11, 2011 at 6:10 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..
> Doctest runs on .rst files, which are plain text files, finds and reports errors, and reports no errors when the errors are fixed.
> See #10875 where is was very helpful. So your last comment puzzles me.
>

Sphinx doctest runner supports doctest fixtures that plain doctest
does not.  Some of the examples in the docs rely on these.

See http://sphinx.pocoo.org/ext/doctest.html .
msg126063 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-12 01:02
Terry, I'm unhappy with the changes to the sorting how-to.  IMO, it was a not a net win (transforming code that already ran fine in something doctest would swallow).  The code snippets now have the visual clutter of the ">>>" and "..." PS1 and PS2 prompts and they can no longer be readily cut and pasted into the interpreter so that people can experiment with them.
msg126067 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-01-12 03:03
There are two reasons I forward ported the changes.

1. Without running doctest on doc examples, they sometimes have errors either originally, after patches to the doc, or after patches to Python. On other issues, I found 4 errors in the json doc (probably original; my fixes were augmented and committed by GB, I believe), and several in the re howto (due to non-update after re changes). There have been other error reports on the tracker and I presume more (other than missing '...') shown in AB's patch. (I just started with the first file changed in the patch.) So I thought it pretty well settled that getting doc examples correct (and test-ready) is be a good idea.

2. Examples with output always start with '>>> ' to differentiate input from output. I thought it pretty standard that continuation lines start with '... '. This is true of all applicable examples in re howto, module chapters for difflib, json, bisect, and 4 of 5 applicable examples in the Built-in Types chapter. Other chapters for binhex, itertools, and tkinter have no multiline interactive examples, so provide no precedent either way.

What is the alternative? The current sorting howto is not consistent:

For some examples, '... ' is just deleted:
>>> student_tuples = [
    ('john', 'A', 15),
    ('jane', 'B', 12),
    ('dave', 'B', 10),
]

For others, '... ' is replaced with '    ':
>>> class Student:
        def __init__(self, name, grade, age):
            self.name = name
            self.grade = grade
            self.age = age

The cmp_to_key function def is a special case: there is no output in the same box with the def and so (normally) should not have prompts. The only reason I followed AB's patch here and added them is because the definition is used in the next box. If the next example were changed for 3.2 to use the new-in-3.2  functools.cmp_to_key(), then the def would not be needed.  (I already planned to suggest thisOr the followup example could just be marked SKIP. With either change, I would remove the prompts added to this function, which are the ones I presume bother you the most.


Summary: according to my current (and subject-to-update ;-) understanding of precedent and policy, the changes other than for cmp_to_key seem correct. The rationale for cmp_to_key changes could easily be negated, in which case they should be reverted.

I will put my thoughts on cut-and-paste in a separate message.
msg126073 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-01-12 04:21
You're welcome to discuss this with me on IRC at some point.  For now, the important thing is that I put a good deal of time and effort working on the sorting howto and I like it as it stands (your notions of precedent or consistency not withstanding).
msg126074 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-12 04:32
On Tue, Jan 11, 2011 at 10:03 PM, Terry J. Reedy <report@bugs.python.org> wrote:
..
> What is the alternative?

I am attaching an alternative patch for sorting howto. You can run the
doctests in it as follows:

$ python2.7 tools/sphinx-build.py -b doctest -d build/doctrees .
build/doctest howto/sorting.rst
...

Doctest summary
===============
   34 tests
    0 failures in tests
    0 failures in setup code
build succeeded.

Unfortunately, Sphinx does not support Ramond's preferred style, but
my patch comes close.
msg126107 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-12 15:33
I have updated Sphinx and rerun py3k doctests.  It looks like the newest Sphinx does not check examples unless the file has a .. testsetup::  directive somewhere.  This is nice because it focuses on the files that are clearly intended to be doctest-checked.  Of course, there are many perfectly good code examples that are now skipped, but this can be addressed in the next step.

I am attaching a patch that makes the following command succeed in py3k Doc directory:

$ sphinx-build --version
Sphinx v1.1pre
..
$ sphinx-build -b doctest -d build/doctrees . build/doctest
..
Doctest summary
===============
  581 tests
    0 failures ..

Most of the changes in issue10225-py3k.diff are non-controvercial with the exception of code fixes in collections and difflib.

In collections, I removed trailing whitespace from generated namedtuple code and in difflib I changed get_close_matches() return value from map object to a list.   I would appreciate comments on these two changes.
msg126111 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-01-12 16:42
Committed the simple fixes in r87957.  Updated the patch to include only the remaining changes.
msg131794 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-03-22 22:23
These are fine except for the changes to collections.py.  Please leave the deployed code for named tuple as-is.  Doctest may have issues with trailing whitespace, but that is doctest's problem, not named tuple's.
msg132019 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-03-24 19:03
> Please leave the deployed code for named tuple as-is.  Doctest may
> have issues with trailing whitespace, but that is doctest's problem,
> not named tuple's.

I am curious, what was the reason to add trailing whitespace in the named tuple template?
msg132020 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2011-03-24 19:15
> I am curious, what was the reason to add 
> trailing whitespace in the named tuple template?

To make it hard to doctest ;-)

I had a thought that it made the template more readable, but the better solution was to just use real newlines instead of '\n'.   The template has been considerably beautified for python 3.3, but I don't want to go back in time and muck with stable code just to accommodate adding a doctest in old docs.
msg222164 - (view) Author: Andy Maier (andymaier) * Date: 2014-07-03 09:06
Hi, I would like to revive this issue and have added a review comment to issue10225-py3k.diff.

Otherwise, I have reviewed the changes in both diffs and think they are good to go.

Andy
msg226121 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-30 04:23
For whatever difference it makes, the 3.x docs now *are* being built with 3.x.
History
Date User Action Args
2016-07-16 01:27:06belopolskysetassignee: belopolsky ->
2016-07-13 05:23:46berker.peksagsetmessages: - msg270297
2016-07-13 05:23:27berker.peksagsetnosy: + georg.brandl, rhettinger, terry.reedy, belopolsky, ezio.melotti, eric.araujo, docs@python, lukasz.langa, andymaier, - lissacoffeyx
2016-07-13 05:15:46lissacoffeyxsetnosy: + lissacoffeyx, - georg.brandl, rhettinger, terry.reedy, belopolsky, ezio.melotti, eric.araujo, docs@python, lukasz.langa, andymaier
messages: + msg270297
2014-08-30 04:23:24terry.reedysetmessages: + msg226121
versions: + Python 3.4, Python 3.5, - Python 3.1, Python 3.2, Python 3.3
2014-07-03 09:06:58andymaiersetnosy: + andymaier
messages: + msg222164
2012-11-17 16:21:11brett.cannonunlinkissue10224 dependencies
2011-05-28 10:09:02ezio.melottisetnosy: + ezio.melotti
2011-03-24 19:15:36rhettingersetmessages: + msg132020
2011-03-24 19:03:51belopolskysetmessages: + msg132019
2011-03-22 22:23:41rhettingersetversions: + Python 3.3
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg131794

assignee: rhettinger -> belopolsky
resolution: accepted
2011-01-12 16:43:07belopolskysetfiles: - issue10225-py3k.diff
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
2011-01-12 16:42:41belopolskysetfiles: + issue10225-py3k.diff
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg126111
2011-01-12 15:38:42belopolskysetfiles: - issue10225-py3k.diff
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
2011-01-12 15:38:35belopolskysetfiles: + issue10225-py3k.diff
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
2011-01-12 15:33:26belopolskysetfiles: + issue10225-py3k.diff
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg126107
2011-01-12 04:36:43rhettingersetassignee: belopolsky -> rhettinger
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
2011-01-12 04:32:02belopolskysetfiles: + sorting-howto.diff

messages: + msg126074
nosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
2011-01-12 04:21:48rhettingersetnosy: georg.brandl, rhettinger, terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg126073
2011-01-12 03:03:52terry.reedysetnosy: + georg.brandl
messages: + msg126067
2011-01-12 01:02:03rhettingersetnosy: + rhettinger
messages: + msg126063
2011-01-11 23:17:06belopolskysetnosy: terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg126051
2011-01-11 23:10:06terry.reedysetnosy: terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg126050
2011-01-10 17:37:27belopolskysetnosy: terry.reedy, belopolsky, eric.araujo, docs@python, lukasz.langa
messages: + msg125906
2011-01-09 20:52:49terry.reedysetnosy: + terry.reedy
messages: + msg125860
2010-11-08 16:12:43belopolskylinkissue10224 dependencies
2010-11-08 15:26:13lukasz.langasetnosy: + lukasz.langa
2010-10-29 16:28:05belopolskysetmessages: + msg119917
2010-10-29 15:40:55belopolskysetfiles: + issue10225a-r27.diff

messages: + msg119908
stage: needs patch -> patch review
2010-10-29 07:50:32belopolskysetnosy: + docs@python
2010-10-29 07:49:47belopolskysetfiles: + issue10225-r27.diff
keywords: + patch
2010-10-29 07:19:05belopolskysetmessages: + msg119869
versions: + Python 3.1, Python 2.7
2010-10-29 02:22:50belopolskysetmessages: + msg119842
2010-10-29 02:20:49eric.araujosetnosy: + eric.araujo
messages: + msg119841
2010-10-29 02:19:06belopolskysetdependencies: + Build 3.x documentation using python3.x
2010-10-29 02:18:53belopolskycreate