classification
Title: Run Tkinter tests with wantobjects=False
Type: enhancement Stage: needs patch
Components: Tests, Tkinter Versions: Python 3.5
process
Status: open Resolution:
Dependencies: 22236 Superseder:
Assigned To: serhiy.storchaka Nosy List: Lita.Cho, gpolo, jesstess, serhiy.storchaka, terry.reedy, zach.ware
Priority: normal Keywords: needs review, patch

Created on 2014-05-26 20:53 by serhiy.storchaka, last changed 2014-08-23 19:49 by zach.ware.

Files
File name Uploaded Description Edit
test_wantobj.patch Lita.Cho, 2014-05-28 23:45 review
test_ttk_guionly_wantobj.py Lita.Cho, 2014-05-28 23:46
wantobj_test.patch Lita.Cho, 2014-08-06 19:04 review
Messages (11)
msg219178 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-05-26 20:53
The wantobjects attribute of the tkinter module is True by default, and all Tkinter tests run with wantobjects=True. We need to test Tkinter with wantobjects=False too. All Tkinter tests should be ran twice, in both mode.
msg219322 - (view) Author: Lita Cho (Lita.Cho) * Date: 2014-05-28 23:44
So I don't know what the best way to do this, but I changed the widget_tests.py in order to set tkinter.wantobjects = 0 if a 'wantobjects' flag was passed through test.support.use_resources. 

Then I added a new test called test_ttk_guionly_wantobj, where it turns on wantobjects. If we create a new module, then we start off with a clean tkinter object.

Before I go this route for all the tests, I wanted to make sure if this was the correct way to go about it.
msg219323 - (view) Author: Lita Cho (Lita.Cho) * Date: 2014-05-28 23:45
Patches lived in my Linux machine. I've attached my patch. I will add my module next.
msg224001 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-07-25 20:56
Thanks Lita, but I think it would be better if all Tkinter tests will automatically run in both wantobjects=0 and wantobjects=1 modes.

As for your patch, ttk.Button() creates default root with current wantobjects value, this default root can be unintentionally used in tests with other wantobjects value. We should ensure that default root is not used in tests.
msg224962 - (view) Author: Lita Cho (Lita.Cho) * Date: 2014-08-06 19:04
Hi Serhiy,

This patch was made while I was learning tkinter. I figured out how to run the tests twice while changing wantobjects variable without creating new tests. Fortunately, all the tests seem to fast when wantobjects is 0 or 1.

The only annoying thing is that it doesn't merge the count of the amount of tests ran within that module, since I am calling support.run_unittest twice. Hopefully, that's okay. Otherwise, I need to figure out how to call the tests twice individually within the generator.
msg225478 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-18 00:00
My impression is that we are trying to move away from using test_main, or is using support.run_unittest considered a sufficient change from the old explicit suite method?

Anyway, here is an alternate approach, adding two lines to wrap all test classes, that tags the test classes in the code and hence in error messages.

import unittest
for i in range(2): exec('''
class T{0}(unittest.TestCase):
    def test_2(self):
        self.assertTrue({0} == 1)
'''.format(i))
unittest.main()
>>>
F.
======================================================================
FAIL: test_2 (__main__.T0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 4, in test_2
AssertionError: False is not true

----------------------------------------------------------------------
Ran 2 tests in 0.016s
msg225617 - (view) Author: Lita Cho (Lita.Cho) * Date: 2014-08-21 20:58
Hi Terry,

I had no idea we were moving away from using test_main. 

So instead, of using support.run_unittest, we should import all the unittest from tkinter/test/ and wrap everything with that exec method, setting wantobjects=1 and again with wantobjects=0?

Also, do you have an example of a unit test that doesn't use test_main? All the unit tests I've seen (tkinter, smtplib, nntplib) all use test_main.



Lita
msg225635 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-08-22 01:54
The current test/test_*.py example file in the docs intentionally does not contain test_main.
https://docs.python.org/3/library/test.html#writing-unit-tests-for-the-test-package
The next section explaining that test.regrtest runs unittest.TestLoader.loadTestsFromModule if test_main does not exist. I believed support.run_unittest.

I never used test_main in test_idle.py, Grep test/test_*.py for 'test_main' or its absence to count others. Or search all issues for 'unittest discovery' for conversions.  For instance: #18258 made this change to the five test_codecmaps*.py files.
http://bugs.python.org/file30644/test_codecmaps_discovery.diff
-def test_main():
-    support.run_unittest(__name__)
-
 if __name__ == "__main__":
-    test_main()
+    support.use_resources = ['urlfetch']
+    unittest.main()

This all said, running all tests in a file with two different values of an attribute of the tested module is an exceptional case.  Zach, what do you think?
msg225642 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-08-22 03:18
I've been thinking on this for a while, and haven't come up with a solution that I like.  I haven't had a chance to look at #22236 yet, but I suspect it will help immensely.

Also, I have a work-in-progress patch for rearranging the tkinter tests into a test.test_tkinter subpackage.  At this point, I'm not sure if that will make this issue easier or harder to resolve, but it does take care of not using test_main for the tkinter tests.  I'm hoping to have some time to get that patch into a submittable state tomorrow, then have a good look at how it, #22236, and this issue interact.  Anyone who wants a look at where I'm going with the test_tkinter subpackage before I get it submitted, it's at http://hg.python.org/sandbox/zware/log/mv_pkg_tests/ (see the changes by doing "hg diff --git -r default -r mv_pkg_tests Lib/test/test_tkinter"; there are changes to other test packages in that branch as well that are missed by specifying Lib/test/test_tkinter).
msg225746 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-23 13:34
Here is my plan. First, we should resolve issue22236. Otherwise some tests will be run with default root created with old "wantobjects" value. Then we should introduce new test resource name as in Lilo's first patch (but "wantobjects" is not correct name, "testallwantobjectsvalues" is more correct but is too long). Then we should add the load_tests() function in every test file which will generate new test classes for every wantobjects value (0, 1 and 2 if issue22214 will be applied) when this resource is enabled.

Any propositions about new resource name?

As for rearranging the tkinter tests, lets discuss this in separate issue. It looks orthogonal to this issue.
msg225759 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-08-23 19:49
I agree that #22236 should be resolved first.  As for resource guarding the 'wantobjects = 0' tests, do we really need to?  It takes a bit under 20 seconds to run all of the tkinter tests on this machine, and I don't think 40 seconds is an unduly long runtime for such a large set of tests, so I would vote for just loading both sets of tests every time.  Or if others feel the 'wantobjects' test should be guarded, I think we could just reuse the 'cpu' resource (since it's really just time that we're guarding).

And the rearrangement will of course be a separate issue; I only mentioned it here because that patch takes care of moving away from test_main and support.run_unittest, and may have an impact on how this issue can be resolved.
History
Date User Action Args
2014-08-23 19:49:02zach.waresetmessages: + msg225759
2014-08-23 13:34:12serhiy.storchakasetmessages: + msg225746
stage: patch review -> needs patch
2014-08-22 03:18:25zach.waresetmessages: + msg225642
2014-08-22 01:54:10terry.reedysetmessages: + msg225635
2014-08-21 20:58:30Lita.Chosetmessages: + msg225617
2014-08-21 19:40:02serhiy.storchakasetdependencies: + Do not use _default_root in Tkinter tests
2014-08-18 00:00:14terry.reedysetnosy: + terry.reedy
messages: + msg225478
2014-08-17 14:26:32serhiy.storchakalinkissue22214 dependencies
2014-08-06 19:04:58Lita.Chosetfiles: + wantobj_test.patch

messages: + msg224962
2014-08-06 16:06:11serhiy.storchakasetassignee: serhiy.storchaka
2014-07-25 20:56:48serhiy.storchakasetmessages: + msg224001
2014-07-13 17:21:23jesstesssetkeywords: + needs review
stage: needs patch -> patch review
2014-05-28 23:46:33Lita.Chosetfiles: + test_ttk_guionly_wantobj.py
2014-05-28 23:45:48Lita.Chosetfiles: + test_wantobj.patch
keywords: + patch
messages: + msg219323
2014-05-28 23:44:14Lita.Chosetmessages: + msg219322
2014-05-28 18:50:04Lita.Chosetnosy: + jesstess, Lita.Cho
2014-05-26 20:53:59serhiy.storchakacreate