Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE: test_browser is failed when run twice #75740

Closed
serhiy-storchaka opened this issue Sep 23, 2017 · 5 comments
Closed

IDLE: test_browser is failed when run twice #75740

serhiy-storchaka opened this issue Sep 23, 2017 · 5 comments
Assignees
Labels
3.7 (EOL) end of life tests Tests in the Lib/test dir topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 31559
Nosy @terryjreedy, @serhiy-storchaka
PRs
  • bpo-31559: Remove test order dependence in idle_test.test_browser. #3708
  • [3.6] bpo-31559: Remove test order dependence in idle_test.test_browser. (GH-3708) #3709
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2017-09-23.19:24:53.939>
    created_at = <Date 2017-09-23.12:06:23.493>
    labels = ['3.7', 'expert-IDLE', 'type-bug', 'tests']
    title = 'IDLE: test_browser is failed when run twice'
    updated_at = <Date 2017-09-23.19:24:53.938>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2017-09-23.19:24:53.938>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2017-09-23.19:24:53.939>
    closer = 'terry.reedy'
    components = ['IDLE', 'Tests']
    creation = <Date 2017-09-23.12:06:23.493>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 31559
    keywords = ['patch']
    message_count = 5.0
    messages = ['302786', '302794', '302795', '302796', '302800']
    nosy_count = 2.0
    nosy_names = ['terry.reedy', 'serhiy.storchaka']
    pr_nums = ['3708', '3709']
    priority = 'high'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue31559'
    versions = ['Python 3.6', 'Python 3.7']

    @serhiy-storchaka
    Copy link
    Member Author

    Buildbots failures:

    http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.x/builds/101/steps/test/logs/stdio
    http://buildbot.python.org/all/builders/AMD64%20Windows8.1%20Refleaks%203.x/builds/116/steps/test/logs/stdio

    I don't know why test_browser fails first time, but this provokes running it second time why always fails.

    $ ./python -m test -vuall -m test_browser test_idle test_idle
    ...

    ======================================================================
    FAIL: test_gettext (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/serhiy/py/cpython3.6/Lib/idlelib/idle_test/test_browser.py", line 181, in test_gettext
        self.assertEqual(self.cbt_C1.GetText(), 'class C1')
    AssertionError: 'class C1()' != 'class C1'
    - class C1()
    ?         --
    + class C1

    ======================================================================
    FAIL: test_init (idlelib.idle_test.test_browser.ChildBrowserTreeItemTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/serhiy/py/cpython3.6/Lib/idlelib/idle_test/test_browser.py", line 175, in test_init
        eq(self.cbt_C1.name, 'C1')
    AssertionError: 'C1()' != 'C1'
    - C1()
    ?   --
    + C1

    ======================================================================
    FAIL: test_getsublist (idlelib.idle_test.test_browser.ModuleBrowserTreeItemTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/serhiy/py/cpython3.6/Lib/idlelib/idle_test/test_browser.py", line 141, in test_getsublist
        self.assertEqual(sub1.name, 'C0')
    AssertionError: 'C0(base)' != 'C0'
    - C0(base)
    + C0

    ======================================================================
    FAIL: test_nested (idlelib.idle_test.test_browser.NestedChildrenTest)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/home/serhiy/py/cpython3.6/Lib/idlelib/idle_test/test_browser.py", line 238, in test_nested
        self.assertEqual(actual_names, expected_names)
    AssertionError: Lists differ: ['f0', 'C0(base)', 'f1', 'c1', 'F1', 'C1()', 'f2', 'C2', 'F3'] != ['f0', 'C0', 'f1', 'c1', 'F1', 'C1()', 'f2', 'C2', 'F3']

    First differing element 1:
    'C0(base)'
    'C0'

    • ['f0', 'C0(base)', 'f1', 'c1', 'F1', 'C1()', 'f2', 'C2', 'F3']
      ? ------

    + ['f0', 'C0', 'f1', 'c1', 'F1', 'C1()', 'f2', 'C2', 'F3']

    ----------------------------------------------------------------------

    The test is passed if run it once on my computer.

    @serhiy-storchaka serhiy-storchaka added the 3.7 (EOL) end of life label Sep 23, 2017
    @serhiy-storchaka serhiy-storchaka added topic-IDLE tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels Sep 23, 2017
    @terryjreedy
    Copy link
    Member

    browser.transform_children, when called the first time on a dict of children nodes, rebinds the name attribute of Classes with bases*. Tests that called the real browser.transform_children the first time on a node mutate the global mock_pyclbr_tree. When the tests are re-run in the same process, tests initially run before the mutation are also run after the mutation.

    *This is not an issue in current real use. IDLE gets a new tree each time a module browser is opened. The current tree widget calls GetSublist and hence transform_children at most once per node as it caches the result for each node. But don't know what ttk.Treeview does and do know that tests can make multiple calls.

    Restoring the tree with
    def tearDownModule():
    C0.name = 'C0'
    C1.name = 'C1'
    fixes the failures found when rerunning the test module in the same process, but not the underlying problem of order dependence.

    Instead, I added immediate calls to transform_children so that the mock tree is never mutated thereafter by any of the tests. The subsequent class checks that the initial name replacements are correct. All tests see the tree in the same state, so there should not be any order dependence left.

    Testing global function calls in a testcase is slightly awkward, but the testcase follows immediately, and making the tree and its nodes global simplifies the tests that use the tree.

    I checked that running test_pyclbr and test_idle twice each succeeds.

    @terryjreedy
    Copy link
    Member

    I don't know why test_browser fails first time,

    On both buildbots, the initial rerun passes:
    Ran 371 tests in 12.265s OK (skipped=3) Gentoo
    Ran 371 tests in 12.047s OK (skipped=1) Windows8

    @terryjreedy
    Copy link
    Member

    New changeset 99167f8 by Terry Jan Reedy in branch 'master':
    bpo-31559: Remove test order dependence in idle_test.test_browser. (bpo-3708)
    99167f8

    @terryjreedy
    Copy link
    Member

    New changeset 429b3b1 by Terry Jan Reedy (Miss Islington (bot)) in branch '3.6':
    [3.6] bpo-31559: Remove test order dependence in idle_test.test_browser. (GH-3708) (bpo-3709)
    429b3b1

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life tests Tests in the Lib/test dir topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants