classification
Title: doctest handling of multiline strings is broken
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Filip Rembiałkowski, serhiy.storchaka, steven.daprano
Priority: normal Keywords:

Created on 2020-04-15 20:37 by Filip Rembiałkowski, last changed 2020-04-16 02:46 by steven.daprano. This issue is now closed.

Files
File name Uploaded Description Edit
doctest-bugs.py Filip Rembiałkowski, 2020-04-15 20:37 isolated test case
doctest-bugs-2.py Filip Rembiałkowski, 2020-04-15 23:15 simplified test case to reproduce the bug
Messages (6)
msg366554 - (view) Author: Filip Rembiałkowski (Filip Rembiałkowski) * Date: 2020-04-15 20:37
The doctest module does not compare multiline strings properly, as attached example proves. 

Tested on 2.7, 3.6 and 3.9.0a5+. (platform: Ubuntu 18.04).

Related: https://stackoverflow.com/questions/60956015/unexpected-errors-while-testing-python3-code-with-doctest
msg366560 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-04-15 21:39
Did you try to run the tested function in the interactive Python interpreter? Did you get what you expected?
msg366567 - (view) Author: Filip Rembiałkowski (Filip Rembiałkowski) * Date: 2020-04-15 22:52
@Serhiy, Thank you for feedback. 

Yes the "testme" function (indeed trivial) works as expected - both in interactive Python interpreter and in script file. 

If you go to Lib/doctest.py, search for "string-identical" and debug my example there, you will see the problem is real.

I don't see an easy fix for the problem, I only suspect it's related to io.StringIO getvalue() method.
msg366569 - (view) Author: Filip Rembiałkowski (Filip Rembiałkowski) * Date: 2020-04-15 23:15
Actually, the behavior does not depend on leading spaces, and test case can be isolated even further. Sample is attached [doctest-bugs-2.py].
msg366574 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-04-16 02:30
Have you tried calling multiline_output() in the REPL?

It does *not* show your expected output:

    # expected
    First line
    Second line


but the string repr():

    # actual
    'First line\nSecond line\n'



Change your doctest to either:


    >>> multiline_output()
    'First line\\nSecond line\\n'

(note that you must escape the backslashes) or:

    >>> print(multiline_output())
    First line
    Second line
    <BLANKLINE>


Note that the "<BLANKLINE>" needs to be written literally, as described here:

https://docs.python.org/3/library/doctest.html#doctest.DONT_ACCEPT_BLANKLINE
msg366575 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-04-16 02:46
By the way Filip, you were told on the Stackoverflow page that the output was correct and that you were not using doctest correctly. Serhiy also hinted to you that you should check the output in the REPL and you falsely claimed that it gave the expected output, but it doesn't:

    py> multiline_output()
    'First line\nSecond line\n'

which is nothing like the output you put in your doctest.

This is not a bug in doctest. The very first line of the doctest documentation says:

    The doctest module searches for pieces of text that look like
    interactive Python sessions

so this is expected and documented behaviour, not a bug.
History
Date User Action Args
2020-04-16 02:46:10steven.dapranosetmessages: + msg366575
2020-04-16 02:30:42steven.dapranosetstatus: open -> closed

nosy: + steven.daprano
messages: + msg366574

resolution: not a bug
stage: resolved
2020-04-15 23:15:57Filip Rembiałkowskisetfiles: + doctest-bugs-2.py

messages: + msg366569
2020-04-15 22:52:50Filip Rembiałkowskisetmessages: + msg366567
2020-04-15 21:39:27serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg366560
2020-04-15 20:37:13Filip Rembiałkowskicreate