msg304163 - (view) |
Author: Denis Osipov (denis-osipov) * |
Date: 2017-10-11 16:55 |
In Developer Guide says:
"If you don’t have easy access to a command line, you can run the test suite from a Python or IDLE shell:
>>> from test import autotest"
But I can't run test from IDLE:
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
from test import autotest
File "D:\repos\cpython\Lib\test\autotest.py", line 5, in <module>
main()
File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 585, in main
Regrtest().main(tests=tests, **kwargs)
File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 510, in main
self._main(tests, kwargs)
File "D:\repos\cpython\Lib\test\libregrtest\main.py", line 524, in _main
setup_tests(self.ns)
File "D:\repos\cpython\Lib\test\libregrtest\setup.py", line 18, in setup_tests
faulthandler.enable(all_threads=True)
io.UnsupportedOperation: fileno
If I understand it correct, this behavior is reasonable (issues 3003 and 25588).
Maybe it's worth to remove words about running from IDLE. Or in case if it's possible to run such tests add some words about it.
|
msg304225 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-10-12 09:51 |
> "If you don’t have easy access to a command line, you can run the test suite from a Python or IDLE shell:
The devguide is wrong. You should not run the Python test suite in IDLE. It doesn't work and many tests fail just because of IDLE.
Please run the test suite in a command line (like "cmd.exe" on Windows).
|
msg304245 - (view) |
Author: Denis Osipov (denis-osipov) * |
Date: 2017-10-12 14:41 |
Got it. Thank you for your help.
|
msg304270 - (view) |
Author: Mariatta (Mariatta) * |
Date: 2017-10-12 17:10 |
Closing this issue, I opened https://github.com/python/devguide/issues/280 so that Dev Guide can be updated.
Thanks!
|
msg304273 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-10-12 17:49 |
See issue25588. Currently running tests from IDLE doesn't work, but it worked in the past and maybe we can make it working again.
|
msg304295 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-12 23:18 |
Victor, what is your opinion of running 'from test import autotest' in a standard python shell window, opened from an icon, as opposed to IDLE's shell?
> many tests fail just because of IDLE.
We cannot know what to blame on IDLE without a proper control experiment. I updated my Win10 repository, rebuilt python 3.7, ran the tests from the command line, and all passed (thank you for making this routine). I ran autotest in a Python console and saw a new resource warning from one of the async tests and 3 failures. I cannot say what they were because when the test finished, the window closed (crashed), which is a metafailure. I will rerun and try to identify the error as they occur.
Should we reopen this issue or open a new issue, to fix problems not due to IDLE, or remove the entire suggestion to use autotest, and remove autotest itself?
|
msg304296 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-12 23:42 |
0:00:28 [ 25/407] test_asyncio
F:\dev\3x\lib\asyncio\sslproto.py:330: ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x066D38C0>
source=self)
test_code_module - multiple errors
test test_importlib failed -- Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_importlib\test_locks.py", line 134, in test_all_locks
self.bootstrap._module_locks)
AssertionError: 0 != 1 : {'test.autotest': <weakref at 0x0395D620; to '_ModuleLock' at 0x03957E70>}
test_warnings failed - multiple errors
crash after summary printed.
I am rerunning in python started from command to see if there is any message printed.
|
msg304298 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-13 00:34 |
The Python console exits because test.libregrtest.main.Regrtest._main calls sys.exit. test.autotest consists of
from test.libregrtest import main
main()
The main call should be wrapped with try:...except sys.exit: pass
The output also has this variance from the command line result.
0:15:47 [319/407/2] test_subprocess
minkernel\crts\ucrt\src\appcrt\lowio\write.cpp(49) : Assertion failed: (_osfile(fh) & FOPEN)
minkernel\crts\ucrt\src\appcrt\lowio\close.cpp(49) : Assertion failed: (_osfile(fh) & FOPEN)
minkernel\crts\ucrt\src\appcrt\lowio\close.cpp(49) : Assertion failed: (_osfile(fh) & FOPEN)
The list of tests skipped seems to be the same.
By setting sys.argv before importing autotest, I got
======================================================================
FAIL: test_ps1 (test.test_code_module.TestInteractiveConsole)
----------------------------------------------------------------------
Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_code_module.py", line 35, in test_ps1
self.assertEqual(self.sysmod.ps1, '>>> ')
AssertionError: <MagicMock name='sys.ps1' id='71914032'> != '>>> '
======================================================================
FAIL: test_ps2 (test.test_code_module.TestInteractiveConsole)
----------------------------------------------------------------------
Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_code_module.py", line 40, in test_ps2
self.assertEqual(self.sysmod.ps2, '... ')
AssertionError: <MagicMock name='sys.ps2' id='72189080'> != '... '
----------------------------------------------------------------------
======================================================================
FAIL: test_missing_filename_main_with_argv (test.test_warnings.CWarnTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_warnings\__init__.py", line 446, in test_missing_filename_main_with_argv
self.assertEqual(w[-1].filename, sys.argv[0])
AssertionError: '__main__' != ''
- __main__
+
and
======================================================================
FAIL: test_missing_filename_main_with_argv (test.test_warnings.PyWarnTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_warnings\__init__.py", line 446, in test_missing_filename_main_with_argv
self.assertEqual(w[-1].filename, sys.argv[0])
AssertionError: '__main__' != ''
- __main__
Wneh I start python with python.bat, sys.argv = [''], not ['__main__'].
|
msg304300 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-13 01:05 |
When I import autotest in IDLE's shell, test_code_module fails, test_importlib and test_warnings fail, as without IDLE, and test_gc and test_logging are new failures. (Results are easy to see because SystemExit is caught and ignored.)
test test_gc failed -- Traceback (most recent call last):
File "F:\dev\3x\lib\test\support\__init__.py", line 1855, in wrapper
return func(*args, **kwargs)
File "F:\dev\3x\lib\test\test_gc.py", line 290, in test_get_count
self.assertLess(a, 5)
AssertionError: 8 not less than 5
<a comes from gc.get_count() after gc.collect(). When I repeat outside of the test, a is 31. I think either gc or this test is buggy.>
test test_logging failed -- Traceback (most recent call last):
File "F:\dev\3x\lib\test\test_logging.py", line 1960, in test_warnings
"dummy.py:42: UserWarning: Explicit\n Dummy line\n")
AssertionError: '\nWarning (from warnings module):\n File[57 chars]it\n' != 'dummy.py:42: UserWarning: Explicit\n Dummy line\n'
+ dummy.py:42: UserWarning: Explicit
-
- Warning (from warnings module):
- File "dummy.py", line 42
- Dummy line
? --
+ Dummy line
- UserWarning: Explicit
IDLE replaces warnings.show_warnings. In any case, there are 3 failures in Python's shell versus 4 in IDLE's shell.
|
msg304326 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-13 08:33 |
After starting Python from a command line instead of an icon, importing/running autotest results in the same three failures.
|
msg304647 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-20 02:22 |
I am re-opening because there is a bug, a conflict between doc and code comment and actual behavior, and one which has nothing to do with IDLE. The question is whether to fix it or to delete the invitation to invoke the bug.
The reason the devguide gives an alternative for running tests is so that someone with only installed Python and no command line could still test their installation and even contribute a patch on the tracker.
On 2000-08-23, Tim Peters add the following comment to test.autotest:
# This should be equivalent to running regrtest.py from the cmdline.
# It can be especially handy if you're in an interactive shell, e.g.,
# from test import autotest.
The autotest code was later copied into test.__main__, and both are now:
from test.libregrtest import main
main()
So 'from test import __main__' is the same as importing autotest, and should be the same as importing and running libregrtest
To test the equivalence in the setting where either import might be useful, I opened both python and IDLE from the 3.7.0a2 Windows start icons and imported autotest. The test deterministicly crashes after test_marshal. I call the stoppage a crash because wrapping the import in try: ... except BaseException has no effect. The Python console just vanishes. IDLE restarts.
...
0:05:51 [207/407/5] test_marshal
=============================== RESTART: Shell =========================
>>>
So the claimed equivalence in worse than just not true. There were, I believe, 3 failures in the Python console. The extra 2 in IDLE are due to the #25588 fix guarding against sys.stderr being None is needed in a couple of other places.
In any case, either running tests by importing should be made to work (again, presuming it once did) or deprecated and autotest removed.
|
msg304674 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2017-10-20 19:39 |
Victor, why do you persist in the nearly irrelevant focus on IDLE? As I reported above, failures and crashes happen ***without*** involving IDLE. Serhiy appears to report the same in #31794.
|
msg304679 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-10-20 20:40 |
Sorry, I didn't read you comment. I just saw IDLE, I missed that you wrote
that the bug is unrelated to IDLE. Ooops
|
msg304682 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2017-10-20 20:46 |
Perhaps there are issues specific to IDLE, but we should first fix issues not related to IDLE in #31794.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:58:53 | admin | set | github: 75942 |
2017-10-20 20:46:32 | serhiy.storchaka | set | messages:
+ msg304682 |
2017-10-20 20:40:43 | vstinner | set | messages:
+ msg304679 |
2017-10-20 19:39:56 | terry.reedy | set | messages:
+ msg304674 title: Failures and crashes when running tests by import in IDLE -> Failures and crashes when running tests by import |
2017-10-20 12:35:25 | vstinner | set | title: Failures and crashes when running tests by import. -> Failures and crashes when running tests by import in IDLE |
2017-10-20 02:22:14 | terry.reedy | set | status: closed -> open versions:
- Python 3.8 title: regrtest: faulthandler.enable() fails with io.UnsupportedOperation: fileno when run from IDLE -> Failures and crashes when running tests by import. messages:
+ msg304647
resolution: not a bug -> stage: resolved -> |
2017-10-13 08:33:08 | terry.reedy | set | messages:
+ msg304326 |
2017-10-13 01:05:01 | terry.reedy | set | messages:
+ msg304300 |
2017-10-13 00:34:15 | terry.reedy | set | messages:
+ msg304298 |
2017-10-12 23:42:47 | terry.reedy | set | messages:
+ msg304296 |
2017-10-12 23:18:26 | terry.reedy | set | nosy:
+ terry.reedy messages:
+ msg304295
|
2017-10-12 17:49:05 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages:
+ msg304273
|
2017-10-12 17:10:46 | Mariatta | set | status: open -> closed
nosy:
+ Mariatta messages:
+ msg304270
resolution: not a bug stage: resolved |
2017-10-12 14:41:24 | denis-osipov | set | messages:
+ msg304245 |
2017-10-12 09:51:16 | vstinner | set | messages:
+ msg304225 |
2017-10-12 09:16:23 | vstinner | set | components:
+ Tests, - Documentation title: Possible error in devguide part about tests -> regrtest: faulthandler.enable() fails with io.UnsupportedOperation: fileno when run from IDLE |
2017-10-11 17:18:50 | serhiy.storchaka | set | nosy:
+ vstinner
|
2017-10-11 16:55:04 | denis-osipov | create | |