This issue proposes stress testing idlelib.run by running code that already reports whether results are as expected or not -- the test_suite. Since this one test will take at least as long as the test suite, and will probably need human intervention anyway, it would not be part of test_idle, but would be run separately manually, like htest.
IDLE runs user code in a user process started with subprocess.POpen(executable + 'path/to/idlelib/run.py' + args + user_code_path) (with user_code_path omitted for shell restarts). When run from the editor, the intent is to imitate 'python -i user_code_path'.
On python list, someone reported that running a program using pyqt5 (and qt5) hung on exit while running with python3 (on Ubuntu, no -i) worked. I have asked for more details, including whether it works with -i included. In the meanwhile, I wondered ...
While the run module might not be able to run programs with certain 3rd party imports, can it at least run programs restricted to stdlib imports and well enough behaved to run with python -i. (A programs that, for instance, disables standard i/o streams would fail both in IDLE and with python.) A test would be to at least run all stdlib tests, with the same result, as run otherwise on the same system. The tests should be run in the user process itself and not in additional processes. They could be run either all in one process or each in a new run process.
If successful, I will put code and instructions in a new idlelib/idle_test/rtest.py/txt.
|
The premise of the issue is this: if IDLE is started with 'pythonx', then running 'filex' from a IDLE editor is equivalent to running "python x -i filex" at a command line. For instance, loading test.__main__ into an editor and running should give the same result as
f:/python/dev> 36\pcbuild\win32\python_d.exe -i 36\Lib\test.__main__
However,
Traceback (most recent call last):
File "F:\Python\dev\36\Lib\test\__main__.py", line 2, in <module>
main()
File "F:\Python\dev\36\lib\test\libregrtest\main.py", line 508, in main
Regrtest().main(tests=tests, **kwargs)
File "F:\Python\dev\36\lib\test\libregrtest\main.py", line 446, in main
self._main(tests, kwargs)
File "F:\Python\dev\36\lib\test\libregrtest\main.py", line 458, in _main
setup_tests(self.ns)
File "F:\Python\dev\36\lib\test\libregrtest\setup.py", line 18, in setup_tests
faulthandler.enable(all_threads=True)
io.UnsupportedOperation: fileno
The pseudofile socket wrappers are not real files with a fileno. I should take a better look as sys.stdout and _stdout (and stderr) when connected to a console,
|
As I reported on related #31761, importing autotest on Windows 10 has 3 errors with Python's shell and 4 with IDLE's, with two of those the same as the python shell failures.
IDLE can be detected in at least two ways.
>>> import sys; 'idlelib.run' in sys.modules
True
>>> type(sys.stdout).__name__ == 'PseudoOutputFile'
True
'test.autotest' in sys.modules" would detect that test.autotest is being run, whether in a Python or IDLE shell.
Running test.__main__ from an editor results in screwy behavior. Running just 'import test.autotest' instead gives the same result. I expected at least the latter to have the same result as when run in the shell. Wrong.
First, there is an additional test failure.
0:00:12 [ 18/407] test_aifc
test test_aifc crashed -- Traceback (most recent call last):
File "F:\dev\3x\lib\test\libregrtest\runtest.py", line 163, in runtest_inner
the_module = importlib.import_module(abstest)
File "F:\dev\3x\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 680, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "F:\dev\3x\lib\test\test_aifc.py", line 9, in <module>
import aifc
File "F:\dev\3x\lib\aifc.py", line 254, in <module>
from chunk import Chunk
ImportError: cannot import name 'Chunk' from 'chunk' (F:\Python\mypy\chunk.py)
Next, the tests run normallu up to
:03:18 [ 77/407/1] test_concurrent_futures
Then, while running test_configparser, the user process crashes, IDLE restarts a new user process, and the test suite restarts (in the IDLE process?) from the beginning, sending output to the console, not IDLE, and running each test 5 times. This all is repeatable. The following is the initial part of the console output.
== CPython 3.7.0a1+ (heads/master:8c26a34f93, Oct 14 2017, 19:37:37) [MSC v.1900 32 bit (Intel)]
== CPython 3.7.0a1+ (heads/master:8c26a34f93, Oct 14 2017, 19:37:37) [MSC v.1900 32 bit (Intel)]
== Windows-10-10.0.14393-SP0 little-endian
== cwd: F:\dev\3x\build\test_python_2056
== CPU count: 12
== Windows-10-10.0.14393-SP0 little-endian
== CPython 3.7.0a1+ (heads/master:8c26a34f93, Oct 14 2017, 19:37:37) [MSC v.1900 32 bit (Intel)]
== cwd: F:\dev\3x\build\test_python_6020
== encodings: locale=cp1252, FS=utf-8
== CPU count: 12
== Windows-10-10.0.14393-SP0 little-endian
Run tests sequentially
== CPython 3.7.0a1+ (heads/master:8c26a34f93, Oct 14 2017, 19:37:37) [MSC v.1900 32 bit (Intel)]
== cwd: F:\dev\3x\build\test_python_272
0:00:00 [ 1/407] test_grammar
== encodings: locale=cp1252, FS=utf-8
== CPU count: 12
== Windows-10-10.0.14393-SP0 little-endian
== CPython 3.7.0a1+ (heads/master:8c26a34f93, Oct 14 2017, 19:37:37) [MSC v.1900 32 bit (Intel)]
Run tests sequentially
== cwd: F:\dev\3x\build\test_python_7396
0:00:00 [ 1/407] test_grammar
== Windows-10-10.0.14393-SP0 little-endian
== encodings: locale=cp1252, FS=utf-8
== CPU count: 12
== cwd: F:\dev\3x\build\test_python_13476
Run tests sequentially
== CPU count: 12
0:00:00 [ 1/407] test_grammar
== encodings: locale=cp1252, FS=utf-8
== encodings: locale=cp1252, FS=utf-8
Run tests sequentially
Run tests sequentially
0:00:00 [ 1/407] test_grammar
0:00:00 [ 1/407] test_grammar
0:00:00 [ 2/407] test_opcodes
0:00:00 [ 2/407] test_opcodes
0:00:00 [ 2/407] test_opcodes
0:00:00 [ 2/407] test_opcodes
0:00:00 [ 2/407] test_opcodes
[and so on]
I said 'crash' because the run process catches SystemExit.
>>> raise SystemExit
>>>
I have ocassionally encountered unexpected restarts before, but never, that I remember, so easily repeatable.
|