Title: IDLE Improvements: Unit test for
Type: enhancement Stage: resolved
Components: IDLE Versions: Python 3.3, Python 3.4, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: JayKrish, Todd.Rovito, kbk, ncoghlan, philwebster, python-dev, roger.serwy, terry.reedy
Priority: normal Keywords: patch

Created on 2013-06-11 11:30 by JayKrish, last changed 2013-06-30 22:39 by terry.reedy. This issue is now closed.

File name Uploaded Description Edit
test_delegator.patch JayKrish, 2013-06-11 11:30 test module for review
test_delegator1.patch JayKrish, 2013-06-17 19:31 requires('gui') Fails. review
test_delegator2.patch JayKrish, 2013-06-24 18:14 requires('gui') does not skip review terry.reedy, 2013-06-24 23:18
Messages (11)
msg190960 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-06-11 11:30
Continuing the IDLE unittest framework initiated in

A small unit test for IDLE module. This patch introduces a test module named inside Lib/idlelib/idle_test, considering the points README file in idle_test. The test method test_setdelegate() simply passes a text widget to Delegator by calling setdelegate(), and test the widget returned by getdelegate() for the same.
msg191241 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-15 22:27
The format looks good.

The content looks skimpy. I am not familiar with Delegator and have not yet looked to see what more is needed. But this is enough to test working with a gui test.

This *is* a gui test. See msg190576. So it must be protected by requiring the gui resource. I am still investigating this subject, but I believe the following should work for the momemnt.

from test.resource import requires
class ...
    def setUpClass(cls):

In a console, "python -m test test_idle" should report that test_delegator.Test_delegator was skipped, follows by other stuff. Please make the change, run from the console, and copy the result here. Add '-ugui' *before* 'test_idle' and Test_delegator should run.

As written, the empty root window is left to be closed by hand. The teardown method should have self.root.destroy(). That will destroy the children also, so I believe that is all that is needed.
msg191367 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-06-17 19:22
The "from test.resource import requires" give me errors saying no module named resource in test (Am I missing anything?)
For the moment I use "from import requires" as in test_delegator1.patch
from import requires
class Test_delegator(unittest.TestCase):

    def setUp(self):

The results follows

jaykrish@jaykrish-Vostro-1520:~/gsoc/cpy/cpython$ ./python -m test -ugui test_idle
[1/1] test_idle
1 test OK.
[189373 refs]

jaykrish@jaykrish-Vostro-1520:~/gsoc/cpy/cpython$ ./python -m test test_idle
[1/1] test_idle
test test_idle failed -- Traceback (most recent call last):
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/unittest/", line 384, in _executeTestPart
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/unittest/", line 32, in testFailure
    raise exception
ImportError: Failed to import test module: idlelib.idle_test.test_delegator
Traceback (most recent call last):
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/unittest/", line 261, in _find_tests
    module = self._get_module_from_name(name)
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/unittest/", line 239, in _get_module_from_name
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/idlelib/idle_test/", line 8, in <module>
  File "/home/jaykrish/gsoc/cpy/cpython/Lib/test/", line 390, in requires
    raise ResourceDenied(msg) Use of the 'gui' resource not enabled

1 test failed:
[189233 refs]

And yes, the teardown method using self.root.destroy() now.
msg191373 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-17 20:22
Yes, 'support', not 'resource'. However, re-read my message for properly using requires to avoid the traceback.

def setUpModule():

may also work, but I am assuming that there might be non-gui test cases added later.
msg191375 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-17 20:40
As I discovered by experiment, unittest.SkipTest and subclasses are only handled properly inside of functions recognized and called by unittest. This could be better documented.
msg191793 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-06-24 18:14
I wrapped requires('gui') as submitted in patch2. But this time commands "python -m test test_idle" and with -ugui does not skip delegatortest (it runs successfully to both).

I took Lib/test/ as an example.
It calls runtktests.get_tests using support.run_unittest from test module.
Exploring Lib/tkinter/test/ the get_tests method begins like

def get_tests(text=True, gui=True, packages=None):
    """Yield all the tests in the modules found by get_tests_modules.
    If nogui is True, only tests that do not require a GUI will be
    attrs = []
    if text:
    if gui:
also notice the main method calles " = ['gui']"
Now the tktests have the line * require('gui') * before the class bigins, for example have a look on 

So do we have to come up with the same procedure to skip and play with gui and non-gui tests for IDLE ? correct me if I am wrong anywhere.
msg191822 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-24 23:18
"" returns True if
"called from a context with __name__ = '__main__'" or
"'gui' in".

The first clause is never true for buildbots, so the presumption is that the test was invoked by a user with a gui screen. (Windoes has an extra test that is only relevant to buildbots.) See #18103 for more discussion.

Here is a minimal file. In what follows, I only note the print output and skip notices.

Running test_gui by itself:

F5 when file is in Idle editor; and
python -m idlelib.idle_test.test_gui
have identical output, as they should,
and print 'in gui', 'nogui', because test_gui is main.

python -m unittest idlelib.idle_test.test_gui
python -m unittest idlelib.idle_test.test_gui.FakeGuiTest
have identical output for this simple file
and print 'nogui', 'skipped=1', because test_gui is not main.

The first of these two lines is not needed; just omit 'unittest'.
The second means that we cannot pick out and run just a gui testcase from a file.

Running as part of suite:

F5 with test_idle loaded in editor window
python -m test -v test_idle
python -m test.test_idle
python -m unittest -v idlelib.idle_test
all print 'skipped', 'nogui', 'skipped=1'
(and otherwise similar output).
because verbosity is set either in the file or command line
and because main is not test_gui and 'gui' not in use_resources.

python -m test test_idle
prints only 'nogui', no skip notice
(because skips are ignored by non-verbose regrtest).

python -m test -ugui test_idle
prints 'in gui', 'nogui'
because regrtest sets use_resources to ['gui'].

python -m idlelib.idle_test does not run but would if there were a .__main__ similar to test.test_idle.

Attribute use_resources can be set by anyone, not just by regrtest.
In test_idle, after "if __name__ == '__main__':", add
    from test import support; support.use_resources = ['gui']
F5 with test_idle loaded in editor window
python -m test.test_idle
both print 'in gui', 'nogui'.

I added and it skips when it should, but one must get verbose output to tell. (I do not see anything on the screen even when it does run.) Perhaps you mistook the absence of a failure report and the absence of a skip report to mean the absence of skips. It does not. Here is what I see.

C:\Users\Terry>python -m test test_idle
[1/1] test_idle
Warning -- os.environ was modified by test_idle
1 test altered the execution environment:

If I add '-ugui', the *only* visible difference is the added 'in gui' from my file. Prints are really useful for leaving a record.

With -v, the difference is
  skipped "Use of the 'gui' resource not enabled"
  test_setdelegate (idlelib.idle_test.test_delegator.Test_delegator) ... ok
It appears that methods within a skipped testcase are not listed, whereas a skipped method within a not-skipped testcase is listed.

I believe your patch is working correctly as far is it goes, and that we should move forward.

The patch needs something like the use_resource line I added to test_idle in order for the gui test to run when is run. The line I gave will work; I am also thinking of a context manager.

As I said on idle-dev, I decided you were right to call your first TestCase 'PathBrowserTest' instead of 'Test_xxx', as I did at first. Having testcases end with 'Test' while methods begin with 'test' is less confusing. So let us make it 'DelegatorTest'.
msg192083 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-30 20:51
The test_delegator patches were useful for working out and testing the use of the gui resource.

However, they are not appropriate for testing the Delegator class. It is a standalone class that has nothing to do with tkinter; in fact, the file has no imports. The test should be a normal text-only test. The only import needed is unittest.

Delegator has two unused methods. We can delete them, leaving just 4 methods to test.
msg192084 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-06-30 20:53
New changeset a568a5426a16 by Terry Jan Reedy in branch '2.7':
Issue 18189: remove unused methods in idlelib.Delegator.Delegator.

New changeset 9d65716367c1 by Terry Jan Reedy in branch '3.3':
Issue 18189: remove unused methods in idlelib.Delegator.Delegator.
msg192088 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-06-30 22:36
I asked on the core-mentorship list about pydev test philosophy. I got useful answers from Nick and Antoine that can be summarized as "We are pragmatic, not dogmatic." One may have to be a list member to read this link, but here it is.

I tried out multiple test methods, but decided that I preferred the continuous scenario in the patch that follows.

Because the code was written before the addition of sets, the private cache was a dict used as a set. I changed it to a set.
msg192089 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-06-30 22:38
New changeset 231c122b44b6 by Terry Jan Reedy in branch '2.7':
Issue #18189: add test_delegator for Idle Delegator class.

New changeset c7605471e8ae by Terry Jan Reedy in branch '3.3':
Issue #18189: add test_delegator for Idle Delegator class.
Date User Action Args
2013-06-30 22:39:33terry.reedysetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2013-06-30 22:38:18python-devsetmessages: + msg192089
2013-06-30 22:36:27terry.reedysetmessages: + msg192088
2013-06-30 20:53:04python-devsetnosy: + python-dev
messages: + msg192084
2013-06-30 20:51:23terry.reedysetmessages: + msg192083
stage: patch review -> needs patch
2013-06-24 23:18:53terry.reedysetfiles: +

messages: + msg191822
stage: patch review
2013-06-24 18:14:46JayKrishsetfiles: + test_delegator2.patch

messages: + msg191793
2013-06-18 02:26:04JayKrishsetnosy: + ncoghlan
2013-06-17 20:40:08terry.reedysetmessages: + msg191375
2013-06-17 20:22:39terry.reedysetmessages: + msg191373
2013-06-17 19:31:22JayKrishsetfiles: + test_delegator1.patch
2013-06-17 19:29:51JayKrishsetfiles: - test_delegator1.patch
2013-06-17 19:22:43JayKrishsetfiles: + test_delegator1.patch

messages: + msg191367
2013-06-15 22:27:01terry.reedysetnosy: + kbk, roger.serwy
messages: + msg191241
2013-06-11 12:56:13Todd.Rovitosetnosy: + terry.reedy
2013-06-11 12:53:08Todd.Rovitosetnosy: + philwebster
2013-06-11 12:47:55Todd.Rovitosetnosy: + Todd.Rovito
2013-06-11 11:45:06JayKrishsetversions: + Python 2.7, Python 3.3
2013-06-11 11:30:19JayKrishcreate