This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: print followed by exception eats print with doctest
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: charlax, cjw296, docs@python, georg.brandl, terry.reedy, tim.peters
Priority: low Keywords:

Created on 2008-08-29 10:54 by cjw296, last changed 2022-04-11 14:56 by admin.

Files
File name Uploaded Description Edit
doctestbug.py cjw296, 2008-08-29 10:55
proof_of_concept_doctest.py charlax, 2008-12-05 11:46 Proof of concept - Doctest bug
doctest_solution.py terry.reedy, 2011-03-09 23:02
Messages (12)
msg72124 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2008-08-29 10:54
Here's an example from a python interpreter session:

Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> def test():
... print "hello"
... raise Exception()
...
 >>> test()
hello
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "<stdin>", line 3, in test
Exception

Now, turning this into a doctests gives:

"""
>>> def test():
...   print "hello"
...   raise Exception()
...
>>> test()
hello
Traceback (most recent call last):
...
Exception
"""

Which when run gives:

**********************************************************************
File "C:\Projects\doctestbug.py", line 6, in __main__
Failed example:
    test()
Exception raised:
    Traceback (most recent call last):
      File "C:\Python25\lib\doctest.py", line 1212, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__[1]>", line 1, in <module>
        test()
      File "<doctest __main__[0]>", line 3, in test
        raise Exception()
    Exception
**********************************************************************
1 items had failures:
   1 of   2 in __main__
***Test Failed*** 1 failures.

The problem is that the function prints output before raising the
exception. If I take the printed output away, the doctest passes fine.
However, especially with dummy fixtures common in doctests, that printed
output needs to be seen to test that things are happening as they should
prior to the exception being raised.
msg72125 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2008-08-29 10:55
Here's the full test file.
msg72140 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2008-08-29 14:52
As the doctest docs say,

    Examples containing both expected output and an exception
    are not supported.  Trying to guess where one ends and
    the other begins is too error-prone, and that also makes
    for a confusing test.

Since this is working as designed and as documented, it's not "a bug". 
You could call it a feature request.
msg72490 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2008-09-04 09:45
Out of interest, where are the doctest docs you quoted? I missed that
bit and that disturbs me :-S

I'm not sure documenting a bug and trying to explain it away makes it
any less of a bug, nonetheless, lets leave this one open as a feature
request then ;-)
msg77007 - (view) Author: Charles-Axel Dein (charlax) * Date: 2008-12-05 11:46
This is a bug. This is not a good behavior.

If I would like to temporarily print a variable to see its content, in 
order to debug my code, doctest will eat its output. Thus I will be make 
to use pdb or to use logging, or to get rid of doctest. This will be far 
more complicated.

See attached file.
msg85210 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2009-04-02 15:19
Hey Georg, I agree with the priority, but I'm still not convinced this 
is just a feature request.

The piece of documentation that Tim is referring to is a tiny footnote 
in the doctest docs, and, as I said previously, I'm not sure documenting 
a bug makes it any less of a bug..
msg85746 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-04-07 20:36
It's not really my call.  Tim is the author of the module and classified
it as a feature request.

Why don't you try to come up with a patch? He might even accept it :D
msg130423 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-03-09 05:01
For the purpose of this tracker, a 'bug' (behavior issue) is a discrepancy between doc and behavior. Micro ('bugfix') releases fix such discrepancies, which are all unintentional.

Every feature request addresses what someone considers a 'design bug'. Micro releases are not intended to contain intentional design changes.
So yes, documenting a design decision makes it not a tracker behavior issue, even if you consider that decision a design bug.

That said, I do not see that temporary debugging output belongs in relatively permanent doctests. That will make the doctest fail as soon as the debugging output is turned back off.

A function whose permanent api is to print to stdout as a side-effect and then raise an exception is very unusual. So I think it OK for doctest to not cover such a thing.

Without a real-world use case, I am inclined to close this issue. Even then, a unittest, where output and exception channels are not mixed together, might be a better choice.
msg130475 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-03-09 23:02
Temporary output will break all doctests, not just those with exception traceback. One should fix, disable debug output, and then rerun doctest to make sure fix did not break anything else.

A function that prints and raises *can* be tested as by separately testing both output and silencing of expected exception as follows:

>>> def test():
...   print("hello")
...   raise IndexError()
...
>>> try:
...   test()
...   raise BaseException()
... except IndexError:
...   pass
hello

This passes but will not if function prints anything else or does anything other than raise that exception (or a subclass thereof). In practice, catching subclasses should be ok, but if not, the except clause can be expanded to

... except IndexError as msg:
...   if type(msg) is not IndexError:
...     raise BaseException

I am only leaving this open for a possible doc addition showing something like the simpler fix above.
msg131541 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2011-03-20 20:57
Terry,

My original post was not about *temporary* output inserted for debugging, but test mocks and the like which form a permanent part of the test and which output to stdout.

cheers,

Chris
msg131566 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-03-20 23:22
You misunderstood my last response. The first paragraph *dismisses* the case of temporary print (raised by Charles-Axle) as out of scope for doctests and hence this issue. The next ones addresses *your* case of code *permanently* intended to print and raise and gives a workable solution to your example.

After failing to find a simpler example that works with 3.x*, I am now thinking of the following doc fix.

1. Add 'directly' before 'supported' in Tim's first sentence: "Examples containing both expected output and an exception are not supported." 

2. After the next sentence of explanation, add

"""Instead, try something like:

>>> def test():
...   print("hello")
...   raise IndexError()
...
>>> try:
...   test()
...   raise BaseException # Fail if no IndexError
... except IndexError:
...   pass
hello
"""

* I discovered a subtle consequence of the print change: "print 1, 1/0" will print '1' on a line by itself and then a traceback. "print(1,1/0)" just prints a traceback. Something like

class C():
    def __str__(self):
        raise IndexError
print(1, C(), sep='\n')

is required to print a value by itself on a line and then a traceback.
That is no simpler and probably less realistic than test() above.
msg383444 - (view) Author: Chris Withers (cjw296) * (Python committer) Date: 2020-12-20 19:56
@iritkatriel - if Tim thinks this is hard, it probably is hard ;-)
History
Date User Action Args
2022-04-11 14:56:38adminsetgithub: 47972
2020-12-20 19:56:52cjw296setkeywords: - easy
2020-12-20 19:56:30cjw296setmessages: + msg383444
2020-12-19 19:33:50iritkatrielsetkeywords: + easy, - patch
versions: + Python 3.10, - Python 3.3
2011-03-20 23:22:01terry.reedysetkeywords: + patch
nosy: tim.peters, georg.brandl, terry.reedy, cjw296, charlax, docs@python
messages: + msg131566
2011-03-20 20:57:28cjw296setnosy: tim.peters, georg.brandl, terry.reedy, cjw296, charlax, docs@python
messages: + msg131541
2011-03-09 23:02:15terry.reedysetfiles: + doctest_solution.py

assignee: docs@python
components: + Documentation, - Library (Lib)

nosy: + docs@python
messages: + msg130475
stage: needs patch
2011-03-09 05:01:45terry.reedysetnosy: + terry.reedy

messages: + msg130423
versions: + Python 3.3, - Python 3.1, Python 2.7
2009-04-07 20:36:46georg.brandlsetnosy: + georg.brandl
messages: + msg85746
2009-04-02 15:19:34cjw296setmessages: + msg85210
versions: + Python 3.1, Python 2.7, - Python 2.5, Python 2.4
2009-04-02 15:11:58georg.brandlsetpriority: low
type: behavior -> enhancement
2008-12-05 11:46:50charlaxsetfiles: + proof_of_concept_doctest.py
nosy: + charlax
type: enhancement -> behavior
messages: + msg77007
2008-09-04 09:45:10cjw296settype: enhancement
messages: + msg72490
2008-08-29 14:52:22tim.peterssetnosy: + tim.peters
messages: + msg72140
2008-08-29 10:55:06cjw296setfiles: + doctestbug.py
messages: + msg72125
2008-08-29 10:54:12cjw296create