I got a doctest failure that when I wrote the output to a file showed two exactly identical lines reported as being different.  Turning off the fancy diff, I could see trailing whitespace on one of the lines.  It turns out that when a fancy diff is requested, doctest explicitly goes through and strips trailing whitespace from the diff lines returned by difflib.  This seems to me to be obviously incorrect.  There is no clue in the changelog why this was done...this goes back to a massive refactoring of doctest that was done for python 2.4, and the fancy diff was introduced at that point, complete with this strange behavior.

I tried to write a test for this but couldn't get it working in the time I was willing to devote to this (I've switched to NDIFF format, which shows the whitespace error even when the actual whitespace is stripped).  Perhaps testing this via doctest isn't the best idea anyway, since it will be far to easy for the trailing whitespace in the test to get accidentally stripped.  I've attached my work as a diff for reference if someone wants to work on this.

I have patched this as explained by David. Also the tests are working. Another test broke because a missing trailing space. I fixed that too.

This is my first patch :D

Unfortunately I was reminded a few days ago that there is a commit hook that prevents patches containing trailing whitespace from being committed to the repository.  So using doctest to test this isn't going to work.

The alternatives are to write a unit test, or to figure out how to write the doctest such there is no trailing whitespace...which would mean capturing the output of doctest.DoctTestRunner instead of letting it go the console, and then comparing it to an explicitly embedded string.

I can write a unittest for this, where should I write it? a new test file?

I believe you can put it in the test_doctest file, and call it using the appropriate runner from test.support from the test_main function.  I haven't tried it though.

> Unfortunately I was reminded a few days ago that there is a commit hook that prevents patches containing trailing whitespace from being committed to the repository.

Is this still the case?

I tried to figure out writing the test case, but I can't wrap my head around showing the trailing whitespace without putting it anywhere in the code.

Also, can you have a look at the PR, please?

To include trailing whitespace on a line in a doctest, _don't_ use raw strings.  Use a regular string, and include add a (one or more) trailing \x20 instead of a space (for example).  For example:

r"""
>>> print("a ")
a 
"""

where there's a space at the end of the output line is no good.  It's visually impossible to tell what's intended, and the commit hook should reject it.  But this works fine (a regular string and an escape code):

"""
>>> print("a ")
a\x20
"""

Thanks for your suggestion, Tim.

I've fixed the patch and it is ready for review at https://github.com/python/cpython/pull/10639

New changeset cbb16459934eaf29c7c7d362939cd05550b2f21f by Senthil Kumaran (Sanyam Khurana) in branch 'master':
bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (#10639)
https://github.com/python/cpython/commit/cbb16459934eaf29c7c7d362939cd05550b2f21f

New changeset 53cf5f084b01cb16630361be5377047c068d2b44 by Miss Islington (bot) in branch '3.7':
bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (GH-10639)
https://github.com/python/cpython/commit/53cf5f084b01cb16630361be5377047c068d2b44

New changeset 5d9ae8b9df8371dd65514e0d60b561fd37056986 by Senthil Kumaran (Miss Islington (bot)) in branch '3.6':
bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (GH-10639) (#11477)
https://github.com/python/cpython/commit/5d9ae8b9df8371dd65514e0d60b561fd37056986

New changeset 02e33d9567aa4bd612f9f08053acbfd5e68480d0 by Senthil Kumaran (Sanyam Khurana) in branch '2.7':
[2.7] bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (#11482)
https://github.com/python/cpython/commit/02e33d9567aa4bd612f9f08053acbfd5e68480d0

test_doctest starts to fail on AMD64 Windows8.1 Refleaks 2.7:

https://buildbot.python.org/all/#/builders/33/builds/471

It may be related to:

New changeset 02e33d9567aa4bd612f9f08053acbfd5e68480d0 by Senthil Kumaran (Sanyam Khurana) in branch '2.7':
[2.7] bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (#11482)
https://github.com/python/cpython/commit/02e33d9567aa4bd612f9f08053acbfd5e68480d0

Yeah, I confirm that it's a regression caused by commit 02e33d9567aa4bd612f9f08053acbfd5e68480d0:

"./python -m test test_doctest test_doctest" fails after this commit, but pass before this commit.

Hi Victor,

Senthil reverted the backport commit on 2.7 branch.

Is there a way I can test reference leaks in CPython? Is it using valgrind or some other tool? Can you please point me to that?

I can then see if I can fix this on 2.7

Thanks!

In short, this buildbot runs "./python -m test -R 3:3 test_doctest". But
here the bug is just that running the test twice in a row fails.

Oh. The 3.6, 3.7 and master changes broke the Refleaks buildbots... By the way, why was such bugfix merged into the 3.6 branch which is now in security-only mode?

Please fix this bug ASAP, or I will revert the change.

I made https://github.com/python/cpython/pull/11501 to fix this problem. After my patch:

❯ ./python.exe -m test test_doctest test_doctest
Run tests sequentially
0:00:00 load avg: 1.82 [1/2] test_doctest
0:00:02 load avg: 1.82 [2/2] test_doctest

== Tests result: SUCCESS ==

All 2 tests OK.

Total duration: 5 sec 286 ms
Tests result: SUCCESS

New changeset c5dc60ea858b8ccf78e8d26db81c307a8f9b2314 by Senthil Kumaran (Pablo Galindo) in branch 'master':
bpo-24746: Fix doctest failures when running the testsuite with -R (#11501)
https://github.com/python/cpython/commit/c5dc60ea858b8ccf78e8d26db81c307a8f9b2314

Hi All,

It was my mistake to merge this in into 3.6, I didn't realize 3.6 was in bugfix mode now. Also I went by the versions (previously) set in this bpo issue.

For the regression caused in refleaks, I think, Pablo's patch will fix it, and I am verifying it here - https://buildbot.python.org/all/#/builders/1/builds/468

This will be backported to 3.7 ( I notice viktor triggered already: https://github.com/python/cpython/pull/11505) and I will cherrpick this to 2.7.

Finally, since the the port to 3.6 was a mistake, I will revert that.

This will make sure that bug is fixed all the supported versions, target branches tests are successful and 3.6 remains unaffected.

> Finally, since the the port to 3.6 was a mistake, I will revert that.

I wrote PR 11499 but I closed it because PR #11501 has been merged, but yeah, maybe a revert is the best option for 3.6.

No problem, the status change of the 3.6 branch is very new, and not everybody got the memo ;-)
https://devguide.python.org/#status-of-python-branches

See this discussion:
https://discuss.python.org/t/remove-needs-backport-to-3-6-label/563

New changeset 1cbd17c6987afc48c16caa7ccc7d19b01fbd39f2 by Miss Islington (bot) in branch '3.7':
bpo-24746: Fix doctest failures when running the testsuite with -R (GH-11501)
https://github.com/python/cpython/commit/1cbd17c6987afc48c16caa7ccc7d19b01fbd39f2

New changeset d09e8cecf214b1de457feae01860f5592f912a8e by Ned Deily (Senthil Kumaran) in branch '3.6':
Revert "bpo-24746: Avoid stripping trailing whitespace in doctest fancy diff (GH-10639) (GH-11477)" (GH-11509)
https://github.com/python/cpython/commit/d09e8cecf214b1de457feae01860f5592f912a8e

New changeset 0167c08163f44f4a033497102244bbb6150f606b by Senthil Kumaran in branch '2.7':
bpo-24746: Fix doctest failures when running the testsuite with -R (#11501) (#11512)
https://github.com/python/cpython/commit/0167c08163f44f4a033497102244bbb6150f606b

All changes related to this issue are done.

Thanks for your contribution and engagement everyone!

All Refleaks buildots are back to green, thanks ;-)