This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: [2.7] regrtest announces 401 tests, but there are 304 tests?
Type: Stage: resolved
Components: Tests, Windows Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: paul.moore, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: buildbot

Created on 2017-05-03 10:44 by vstinner, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)
msg292886 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 10:44
First case:
---
...
[301/401] test_with
[302/401] test_import
[303/401] test_xmlrpc
277 tests OK.
2 tests altered the execution environment:
    test_distutils test_sax
24 tests skipped:
    test_al test_bsddb185 test_cl test_commands test_curses test_dbm
    test_dl test_fork1 test_grp test_ioctl test_kqueue test_macos
    test_openpty test_ossaudiodev test_pipes test_poll test_posix
    test_pwd test_readline test_scriptpackages test_spwd
    test_threadsignals test_wait3 test_wait4
1 skip unexpected on win32:
    test_readline
---
http://buildbot.python.org/all/builders/AMD64%20Windows10%202.7/builds/133/steps/test/logs/stdio

Why does regrtest count 404 tests, whereas the total is 304 tests (277+2+24+1)?

Second case:
---
...

[212/401] test_extcall
[213/401] test_frozen
[214/401] test_hotshot
190 tests OK.
2 tests altered the execution environment:
    test_distutils test_sax
22 tests skipped:
    test_al test_applesingle test_bsddb185 test_crypt test_curses
    test_dbm test_dl test_fcntl test_fork1 test_gdb test_gl test_mhlib
    test_nis test_openpty test_ossaudiodev test_pipes test_poll
    test_pwd test_resource test_scriptpackages test_threadsignals
    test_wait4
1 skip unexpected on win32:
    test_gdb
---
http://buildbot.python.org/all/builders/AMD64%20Windows8%202.7/builds/128/steps/test/logs/stdio

Here I only count 215 tests (190+2+22+1)... why not 304 or 401? where are the remaining tests?


On Linux, it seems fine:
---
...
[399/401] test_sets
[400/401] test_bufio
[401/401] test_xpickle
371 tests OK.
30 tests skipped:
    test_aepack test_al test_applesingle test_bsddb test_bsddb185
    test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_gdb
    test_gl test_imgfile test_kqueue test_macos test_macostools
    test_msilib test_py3kwarn test_scriptpackages test_startfile
    test_sunaudiodev test_tk test_tools test_ttk_guionly test_winreg
    test_winsound test_zipfile64
9 skips unexpected on linux2:
    test_bsddb test_bsddb3 test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_gdb
    test_tools
---
http://buildbot.python.org/all/builders/x86%20Gentoo%20Installed%20with%20X%202.7/builds/129/steps/test/logs/stdio

I also expects regrtest to ends with "[401/401]" on Windows.
msg292887 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 10:45
Another example, worse:
---
...
[123/401] test_source_encoding
[124/401] test_locale
[125/401] test_unary
116 tests OK.
9 tests skipped:
    test_applesingle test_bsddb185 test_ioctl test_kqueue
    test_linuxaudiodev test_msilib test_startfile test_sunaudiodev
    test_tk
1 skip unexpected on linux2:
    test_ioctl
---
http://buildbot.python.org/all/builders/PPC64LE%20Fedora%202.7/builds/127/steps/test/logs/stdio

Only 116 tests here???
msg292895 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 12:28
Maybe be related to issue #30255 (test_xml_etree crash related to slices).
msg292905 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-03 14:31
It has been long time since I didn't work on Python 2.7, so I forgot that regrtest of Python 2.7 lacks *many* features of Python master. I created the issue #30258 to add code to handle crash of a test child process.

> Maybe be related to issue #30255 (test_xml_etree crash related to slices).

regrtest stopped at the first crash of a child process and swallow child stdout/stderr. This crash explains this issue. Since issue #30255 is fixed, I close this issue to now focus on the issue #30258 (fix regrtest to detect crashes).
History
Date User Action Args
2022-04-11 14:58:46adminsetgithub: 74440
2017-05-03 14:31:53vstinnersetstatus: open -> closed
resolution: fixed
stage: resolved
2017-05-03 14:31:42vstinnersetmessages: + msg292905
2017-05-03 12:28:32vstinnersetmessages: + msg292895
2017-05-03 10:45:04vstinnersetmessages: + msg292887
2017-05-03 10:44:11vstinnercreate