Test installers before releasing (ModuleNotFoundErrors) #84931
Comments
|
https://www.python.org/ftp/python/3.9.0/python-3.9.0b1-amd64.exe C:\Python39>python.exe -m test test_deque -v ====================================================================== Traceback (most recent call last):
File "C:\Python39\lib\test\test_deque.py", line 781, in test_sizeof
check(deque(), basesize + blocksize)
File "C:\Python39\lib\test\support\__init__.py", line 1483, in check_sizeof
import _testinternalcapi
ModuleNotFoundError: No module named '_testinternalcapi'Ran 79 tests in 0.696s FAILED (errors=1, skipped=1) == Tests result: FAILURE == 1 test failed: Total duration: 906 ms C:\Python39> |
hongleijiang
mannequin
added
3.9
|
At a guess, it looks like we're not including that module in the Windows installer. And indeed, it looks like it's left out of Tools/msi/test/test_files.wxs if you would like to provide a PR :) On the other hand, I'm not sure that test (and any other depending on that module) shouldn't be skipped if _testinternalcapi is missing. |
|
== Tests result: FAILURE == 357 tests OK. 24 tests failed: 43 tests skipped: Zachary Ware <report@bugs.python.org> 于2020年5月26日周二 上午3:33写道：
|
|
I verified that running tests for installed 64-bit 3.9.0b1 on Win10 is somewhat a joke. I got 22 failures. Multiple tests failed on missing _testinternalcapi, and at least 1 on _xxtestfuzz. Several tests had multiple failures and would need to be run individually and verbosely to see why. For instance, test_itertools tries to run test.support.__init__.checksizeof 4 times, and the latter imports _testinternalcapi, resulting is 4 failures. Release managers: I think installers should be run and the suite run with the result. This is not the first time a file has been omitted. One needed by IDLE was once omitted, I believe on Windows. There is now a test that reads it, but is useless unless run. (And I should add checks for all the other 'not .py' files.) honglei: Thank you for both posts. Next time, please remove the quoted message when replying by email. Once posted on the web page, below the quoted message, the quote becomes redundant noise. |
terryjreedy
changed the title
terryjreedy
changed the title
|
We have a bunch of buildbots that test the installed Python. For instance, if you search for "installed" in https://buildbot.python.org/all/#/builders?tags=%2B3.x we have: aarch64 Fedora Rawhide Clang Installed 3.x So it seems that only Linux has "installed" builbot tests. I suppose the next step is adding some macOS and Windows ones. |
Having more buildbots test from "installed" locations, rather than the build directory, is good. But that's not the whole issue here. Those "installed" tests are only testing that a Python runs from an installed location on the same system it was built. The Windows and macOS installers use various packaging methods to provide complete Python installations, including binaries, that can be installed on other systems using various OS-native install methods familiar to users of each platform. And the binaries are generally built in such a way to support running on multiple release versions of its operating system. For example, the current Python installers for macOS from python.org are usable on macOS 10.9 through 10.15. To produce the various installer packages, there are other layers of tooling involved, say, on top of "make install" in the Unixy build process for macOS; something similar is true for Windows releases. FWIW, every macOS installer is test installed on a vanilla system, the whole standard test suite is run, and an IDLE smoke test is run to ensure there are no noticeable regressions before handing the build artifacts off to the release manager. Every installer is tested on at least one supported macOS version and usually more than one. |
|
There are non-binding CI tests run with Github Actions (post-merge) that test from a layout, and last I checked they were all passing. Nobody is really monitoring it but me though, as it isn't a buildbot. The Windows installer also runs pip and IDLE tests before being published, though not a full test suite. Honestly, it's pretty quick to generate an install layout for Windows now I have the PC/layout script. We could do it for every test run if we wanted, though then a number of build-tree only tests would be skipped. |
That's good and reminds me that I forgot to mention that for the macOS installer, besides the already mentioned tests, we also download, install, and smoke test a third-party wheel from PyPI using the bundled pip and we also do a visual inspection of the release's changelog by launching through IDLE the docset bundled with each installer. If the Python binaries included with the installer are built for multiple CPU architectures (macOS universal binaries), we would also attempt to ensure that the test suite is run at least once on each architecture (as of several months ago, we no longer regularly distribute such binaries but there's a good chance we may need to do so again in the not-too-distant future, for example, if Apple introduces macOS machines based on ARM CPUs). |
|
c:\Python39>python -m test -j4 == Tests result: FAILURE == 358 tests OK. 24 tests failed: 43 tests skipped: Total duration: 6 min 44 sec |
|
Still missing _testinternalcapi module for Python3.9rc2 C:\Users\Admin>c:\python39\python.exe -m test test_deque
0:00:00 Run tests sequentially
0:00:00 [1/1] test_deque
test test_deque failed -- Traceback (most recent call last):
File "c:\python39\lib\test\test_deque.py", line 781, in test_sizeof
check(deque(), basesize + blocksize)
File "c:\python39\lib\test\support\__init__.py", line 1484, in check_sizeof
import _testinternalcapi
ModuleNotFoundError: No module named '_testinternalcapi'test_deque failed == Tests result: FAILURE == 1 test failed: Total duration: 1.1 sec |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: