classification
Title: Enhance doctest support in regrtest --list-cases
Type: enhancement Stage: resolved
Components: Tests Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: haypo, serhiy.storchaka
Priority: normal Keywords:

Created on 2017-06-16 11:40 by haypo, last changed 2017-08-11 00:04 by haypo. This issue is now closed.

Messages (4)
msg296179 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 11:40
bpo-30523 added a new --list-cases command to regrtest to list test methods. It is able to list doctest tests in test_builtins, but not in test_extcall.

test_builtin doctest tests:

haypo@selma$ ./python -m test test_builtin --list-cases|grep ^builtins
builtins.bin
builtins.float.as_integer_ratio
...


test_builtins works because it uses:

def load_tests(loader, tests, pattern):
    from doctest import DocTestSuite
    tests.addTest(DocTestSuite(builtins))
    return tests

Listing test methods of test_extcall doesn't work, the following command has no output (but succeed):

./python -m test test_extcall --list-cases

test_extcall uses:

def test_main():
    support.run_doctest(sys.modules[__name__], True)

I see two options:

* Replace support.run_doctest() with doctest.DocTestSuite() in all tests
* Enhance --list-cases to discover doctests. support.run_doctest() calls doctest.testmod(), but doctest.testmod() has no API to list tests. testmod() lists tests and directly runs them. Maybe --list-cases can reuse doctest.DocTestSuite(), I don't know.
msg296203 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-06-16 15:35
Note that --match (and --matchfile) is ignored by doctests. I think this can cause a problem for bisect feature. After making --match working with doctests we can found a way to name doctests and list them.
msg296205 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-06-16 15:37
> Note that --match (and --matchfile) is ignored by doctests, so technically, it seems that it's doable ;-)

In test_os, --match works on doctest tests:

haypo@selma$ ./python -m test -v test_builtin --match=builtins.hex 
(...)
0:00:00 load avg: 0.28 [1/1] test_builtin
hex (builtins)
Doctest: builtins.hex ... ok

----------------------------------------------------------------------
Ran 1 test in 0.002s
(...)
msg300140 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-11 00:04
I have no idea how to implement this idea. I'm not interested to work on this topic, so I just close this issue. If someone wants to work on this topic, please open a new issue.
History
Date User Action Args
2017-08-11 00:04:07hayposetstatus: open -> closed
resolution: out of date
messages: + msg300140

stage: resolved
2017-06-16 15:37:20hayposetmessages: + msg296205
2017-06-16 15:35:16serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg296203
2017-06-16 11:40:34haypocreate