This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: doctest.DocTestCase fails when run repeatedly
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: JelleZijlstra, PiDelport, amaury.forgeotdarc, danielnoord, iritkatriel, jamesh, mgedmin, miss-islington, python-dev
Priority: normal Keywords: easy, patch

Created on 2008-04-10 03:40 by PiDelport, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
reset_globs.patch PiDelport, 2008-04-19 06:23 Zope's fix, plus a test docstring tweak review
Pull Requests
URL Status Linked Edit
PR 31932 merged python-dev, 2022-03-16 13:35
PR 32057 merged miss-islington, 2022-03-22 21:01
PR 32058 merged miss-islington, 2022-03-22 21:02
Messages (15)
msg65285 - (view) Author: Pi Delport (PiDelport) Date: 2008-04-10 03:40
DocTestCase.tearDown destructively clears its DocTest instance's globs,
preventing the test from being run repeatedly (such as with trial
--until-failure).

There's a fix for this in zope.testing's version of doctest, which
resets the globs instead:

http://svn.zope.org/?view=rev&rev=39023
msg65286 - (view) Author: Pi Delport (PiDelport) Date: 2008-04-10 04:23
Addendum:  This appears to be a regression in r36809 (Python 2.4+).
msg65335 - (view) Author: James Henstridge (jamesh) Date: 2008-04-11 06:26
Is repeating a test with the same TestCase instance ever safe?  It'd be
better to create a new instance and run that.

If any of the variables in test.globs are changed by the test (e.g.
appending to a list), then rerunning the test will not necessarily give
the same result.

While the zope change allows tests that have immutable globals or don't
change their globals to function, it also lets other tests "almost"
work, and could lead to new bugs that are difficult to track down.
msg65339 - (view) Author: Pi Delport (PiDelport) Date: 2008-04-11 08:00
> If any of the variables in test.globs are changed by the test (e.g. 
appending to a list), then rerunning the test will not necessarily give
the same result.

This is true, but modifying the globals such that subsequent runs of the
same test can break equally affects subsequent runs of any other tests
that use that module:  such a test is already broken (unsafe to run with
other tests) to begin with, independent of the DocTestCase.tearDown issue.
msg65344 - (view) Author: James Henstridge (jamesh) Date: 2008-04-11 09:22
If I create a test case with a command like:

   test = DocFileSuite('foo.txt', globs={'somelist': [42]})

The doctest isn't doing anything wrong if it modifies somelist.

Furthermore, Glyph has said he thinks the current --until-failure
behaviour in trial is a mistake: http://glyf.livejournal.com/72505.html
msg65345 - (view) Author: Pi Delport (PiDelport) Date: 2008-04-11 10:29
Well, whether that code is wrong depends on whether your project policy
wants repeatable tests or not.  A repeatable and arguably more idiomatic
way of writing that example is to give DocFileSuite a setUp function
which initializes any special globals required by the test.

In any case, DocTestCase allowing non-repeatable tests (which i don't
think are common) is no reason to disallow repeatable tests, which e.g.
at least Zope considers important enough to motivate fixing this issue.
 (Zope is also the source of the code being fixed, if i'm not mistaken.)
msg65346 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-04-11 11:13
FWIW, the python testsuite needs repeatable tests, when running in
"reference leaks" mode.

See also r62100, where a DocTestSuite construction had to be moved into
the repeated function.
msg167696 - (view) Author: Marius Gedminas (mgedmin) * Date: 2012-08-08 16:39
For the record, this bug also breaks zope.testrunner's --repeat option, if you have any doctests in your test suite that rely on test.globs not going away.
msg415090 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-03-13 19:55
I've reproduced this on 3.11 as well.

The patch here needs to be converted to a GitHub PR and then tested and reviewed.

The patch on issue9736 has a unit test as well, which should be included (because the pjd's patch doesn't have one).
msg415339 - (view) Author: Daniël van Noord (danielnoord) * Date: 2022-03-16 13:37
I have created a PR that combines the patch here with the patch from issue9736.
msg415812 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-03-22 21:01
New changeset 7ba7eae50803b11766421cb8aae1780058a57e2b by Daniël van Noord in branch 'main':
bpo-2604: Make doctest.DocTestCase reset globs in teardown (GH-31932)
https://github.com/python/cpython/commit/7ba7eae50803b11766421cb8aae1780058a57e2b
msg415817 - (view) Author: miss-islington (miss-islington) Date: 2022-03-22 21:27
New changeset 3c6019035f16b673cf0f0be6918f7d5493e5690e by Miss Islington (bot) in branch '3.9':
bpo-2604: Make doctest.DocTestCase reset globs in teardown (GH-31932)
https://github.com/python/cpython/commit/3c6019035f16b673cf0f0be6918f7d5493e5690e
msg415818 - (view) Author: miss-islington (miss-islington) Date: 2022-03-22 21:32
New changeset f163ad22d3321cb9bb4e6cbaac5a723444641565 by Miss Islington (bot) in branch '3.10':
bpo-2604: Make doctest.DocTestCase reset globs in teardown (GH-31932)
https://github.com/python/cpython/commit/f163ad22d3321cb9bb4e6cbaac5a723444641565
msg415849 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-03-23 02:02
After 14 years this bug is finally fixed. Thanks everyone for the patches and discussion.
msg416248 - (view) Author: Pi Delport (PiDelport) Date: 2022-03-29 08:59
Thanks! 😄️
History
Date User Action Args
2022-04-11 14:56:33adminsetgithub: 46856
2022-03-29 09:00:00PiDelportsetmessages: + msg416248
2022-03-23 02:02:51JelleZijlstrasetstatus: open -> closed
resolution: fixed
messages: + msg415849

stage: patch review -> resolved
2022-03-22 21:32:02miss-islingtonsetmessages: + msg415818
2022-03-22 21:27:38miss-islingtonsetmessages: + msg415817
2022-03-22 21:02:00miss-islingtonsetpull_requests: + pull_request30150
2022-03-22 21:01:27miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request30149
2022-03-22 21:01:19JelleZijlstrasetnosy: + JelleZijlstra
messages: + msg415812
2022-03-18 01:59:38JelleZijlstralinkissue9736 superseder
2022-03-16 13:37:03danielnoordsetnosy: + danielnoord
messages: + msg415339
2022-03-16 13:35:32python-devsetnosy: + python-dev

pull_requests: + pull_request30023
stage: patch review
2022-03-13 19:55:29iritkatrielsetversions: + Python 3.11
nosy: + iritkatriel

messages: + msg415090

keywords: + easy
2020-11-01 00:47:17iritkatrielsetversions: + Python 3.9, Python 3.10, - Python 3.1, Python 2.7, Python 3.2
2016-05-28 20:44:12berker.peksaglinkissue9327 superseder
2012-08-08 16:39:48mgedminsetnosy: + mgedmin
messages: + msg167696
2010-06-09 22:12:13terry.reedysetversions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 2.5, Python 2.4, Python 3.0
2008-04-19 06:23:33PiDelportsetfiles: + reset_globs.patch
keywords: + patch
2008-04-11 11:13:43amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg65346
2008-04-11 10:29:03PiDelportsetmessages: + msg65345
2008-04-11 09:22:31jameshsetmessages: + msg65344
2008-04-11 08:00:02PiDelportsetmessages: + msg65339
2008-04-11 06:26:21jameshsetnosy: + jamesh
messages: + msg65335
2008-04-10 04:23:42PiDelportsetmessages: + msg65286
2008-04-10 03:40:45PiDelportcreate