Author terry.reedy
Recipients ezio.melotti, r.david.murray, serhiy.storchaka, terry.reedy, zach.ware
Date 2013-07-20.00:17:44
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1374279467.21.0.876068060763.issue18492@psf.upfronthosting.co.za>
In-reply-to
Content
Currently, all requires() tests pass when the file they occur in is run  as '__main__'. This is especially needed when the file ends with the now standard boilerplate.
  if __name__ == '__main__':
      ...
      unittest.main(...)
as there is currently no way to set resources within the unittest.main call.

The problem is that this permissiveness does not apply to subsidiary files discovered from and run by a main file, even though it should. The current workaround is to explicitly set use_resources for the benefit of subsidiary files, as in test_idle.py.

As I see it, the main point of this patch, somewhat obscured by the title, is to extend the current resource permissiveness from tests *in* main files run as main to tests in other files discovered and run from a main file. It also extends the permisiveness to any test not run by regrtest (ie, by unittest).

The key change is
-    if sys._getframe(1).f_globals.get("__name__") == "__main__":
+    if not regrtest_run:

'regrtest_run == True' is currently spelled 'use_resources is not None', so the latter could be used instead to replace the frame-check without otherwise adding new code. 

Extending the permissiveness from main files to subsidiary files strikes me as a no-brainer. Splitting a (potentially) large file into a master file and a package of subsidiary files should not affect which tests are run.

More interesting is extending the permisiveness to tests run under unittest with "python -m unittest target". Target can be a master file, a test file, or a test case or test methods. Subfile targets can only be run with unittest, not regrtest, and there is no way to enable resources for such targets.
  python -m unittest idlelib.idle_test.test_xy.TextText # runs
  python -m unittest idlelib.idle_test.test_xy.GuiText # skips
So the patch enables something that is currently not possible.

Serhiy is concerned about the possible booby-trap for users that run slow resource intensive tests. Some thoughts:

* Running tests as main is mainly done interactively, and usually by developers at that. Humans can stop a test and ignore errors better than buildbots.

* There are multiple ways to run a file from the command line. The test chapter of the manual can document that 
  python -m test.test_xyz
is more or less equivalent to
  python -m test -uall test_zyz
with -v possibly tossed in. Anyone who does not want that can still run under regrtest by using the currently documented
  python -m test test_xyz

* Anyone running a test file loaded in an Idle window very likely wants to run all tests possible.

* Documenting that running under unittest enables all resources is trickier as long as resources are cpython and regrtest specific. I think I would mention this in the test chapter, where resources are discussed, rather than the unittest chapter.

*If -u is added to unittest (and 'use=' to .main), a default of all would be the right thing for subfile targets, even if not for file and multi-file targets.
---
History
Date User Action Args
2013-07-20 00:17:47terry.reedysetrecipients: + terry.reedy, ezio.melotti, r.david.murray, zach.ware, serhiy.storchaka
2013-07-20 00:17:47terry.reedysetmessageid: <1374279467.21.0.876068060763.issue18492@psf.upfronthosting.co.za>
2013-07-20 00:17:47terry.reedylinkissue18492 messages
2013-07-20 00:17:44terry.reedycreate