classification
Title: test_ttk timeout: FAIL tkinter ttk LabeledScale test_resize, and more
Type: behavior Stage: patch review
Components: Tests, Tkinter Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: doko, epaine, miss-islington, serhiy.storchaka, steve.dower, terry.reedy
Priority: normal Keywords: patch

Created on 2020-10-24 23:29 by terry.reedy, last changed 2020-11-30 08:44 by miss-islington.

Files
File name Uploaded Description Edit
ismapped.py epaine, 2020-11-05 20:54
Pull Requests
URL Status Linked Edit
PR 23156 closed epaine, 2020-11-04 23:06
PR 23474 merged serhiy.storchaka, 2020-11-23 10:19
PR 23565 merged miss-islington, 2020-11-30 08:24
PR 23566 merged miss-islington, 2020-11-30 08:24
Messages (20)
msg379543 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-24 23:29
This appears to have timed out as there was no other error indication.
Azure Pipelines, retested gave 2nd failure.  Passed on another PR.

test_resize (tkinter.test.test_ttk.test_extensions.LabeledScaleTest) ... Timeout (0:20:00)!
Thread 0x00007f787182e080 (most recent call first):
  File "/home/vsts/work/1/s/Lib/tkinter/__init__.py", line 696 in wait_visibility
  File "/home/vsts/work/1/s/Lib/tkinter/test/test_ttk/test_extensions.py", line 190 in test_resize
...
msg379556 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-25 03:09
Azure Pipelines Ubuntu, as before, but Combobox test_virtual_event.

test_virtual_event (tkinter.test.test_ttk.test_widgets.ComboboxTest) ... Timeout (0:20:00)!
Thread 0x00007fb09e3f3080 (most recent call first):
  File "/home/vsts/work/1/s/Lib/tkinter/__init__.py", line 696 in wait_visibility
  File "/home/vsts/work/1/s/Lib/tkinter/test/test_ttk/test_widgets.py", line 452 in test_virtual_event
msg379686 - (view) Author: E. Paine (epaine) * Date: 2020-10-26 20:47
This is strange as these only seem to have started occurring recently (though there hasn't been a change to the tests themselves in a long time). There must be something different about how Azure runs the GUI tests because it was on Azure that the tests on PR-22682 failed (noting that it was only on Azure they failed and all the Azure bots failed - suspicious). Is there someone we can nosy who would know better how Azure works (and hence why the possible difference)?

I am looking into both Azure and the tests themselves but am not really sure how to check if they pass without a messy commit history.
msg379695 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-26 22:29
Steve, we have a new Pipelines-specific intermittent tkinter test failure.  Are you familiar with Pipelines, or do you know who is?
msg379702 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-10-27 00:15
I'm as familiar as anyone else that I'll be able to find, but I've really got no way to explain why the window isn't becoming visible.

Is it possible that it's trying to default to the process's nShowCmd value instead of a constant "normal" window size? That would seem most likely - nearly every issue we've had along these lines in the past have been some form of inherited process-wide setting.

The other possibility is if the screen size is so small that the window is being put outside of the bounds, and then the visibility test being used is one that checks whether its bounds are within the current monitor. That may also get a false negative in some non-interactive cases.

(I'm assuming, based on two examples, that they're all timing out at the "wait_visibility" call.)
msg380371 - (view) Author: E. Paine (epaine) * Date: 2020-11-04 22:11
I was wrong trying to blame Azure. I have now seen this happen on an Ubuntu Github Action for PR-22947 (https://github.com/python/cpython/pull/22947/checks?check_run_id=1354118848).

test_virtual_event (tkinter.test.test_ttk.test_widgets.ComboboxTest) ... Timeout (0:20:00)!
Thread 0x00007f549f9d6080 (most recent call first):
  File "/home/runner/work/cpython/cpython/Lib/tkinter/__init__.py", line 696 in wait_visibility
  File "/home/runner/work/cpython/cpython/Lib/tkinter/test/test_ttk/test_widgets.py", line 452 in test_virtual_event

I am tempted just to remove all calls to `wait_visibility` in our tests.
msg380377 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-11-04 22:52
All 3 of these timeout failures are Ubuntu.  They are a $*#(*&(#& nuisance for a required test.  Please submit a PR to skip on Ubuntu, if we can detect that, or linux, if not.

I have the impression that LabeledScale and Combobox are both composite widgets, in some sense.  Correct?  If so, we might limit the skip to those until the wait fails for something else.
msg380433 - (view) Author: E. Paine (epaine) * Date: 2020-11-05 20:54
I have been looking for a more permanent solution to that proposed in PR-23156. The most obvious solution is probably what I suggested in msg380371 as I don't believe the calls to `wait_visibility` are required (though, I don't really know enough to say for certain so am wary of making the tests unstable in a different way).

Another initial idea I had would be to mess around with threads and interrupt the main thread (which would be caught and ignored) if `wait_visibility` took too long.

It would be better, though, if we could actively predict a `wait_visibility` failure. I looked again at the Tcler's wiki (https://wiki.tcl-lang.org/page/tkwait+visibility) and noticed the hidden discussion titled "Windows bug or whatever - looking for help". In that discussion is a description of exactly the issue we are facing:
"Under strange totally unrelated circumstances [...] tkwait waits forever."

There, the OP includes a version which only calls `wait_visibility` if `winfo_ismapped` is 1, saying:
"The "winfo ismapped" is only 0 if the tkwait will stall"

In some initial testing (see attached demo script - uncomment the update call for `wait_visibility` to stall) I found the opposite to be true (`wait_visibility` fails when the widget *is already* mapped - I think this is because `wait_visibility` is waiting for VisibilityNotify but this was already issued when the widget was being mapped). Another proposed solution on the wiki is to avoid `wait_visibility` completely and instead use a combination of various calls including `tkraise` & `focus_force` (ideas/opinions?)

Note: I included both `winfo_ismapped` and `winfo_viewable` in the test script to try and find a case where the value of one is not the same as the other but I believe `winfo_ismapped` is the one we would want to use.
msg380555 - (view) Author: E. Paine (epaine) * Date: 2020-11-08 15:56
This is starting to get *very* annoying. Today's failure was a new one (but still `wait_visibility`):

test_variable_change (tkinter.test.test_ttk.test_extensions.LabeledScaleTest) ... Timeout (0:20:00)!
Thread 0x00007f31dade1080 (most recent call first):
  File "/home/runner/work/cpython/cpython/Lib/tkinter/__init__.py", line 696 in wait_visibility
  File "/home/runner/work/cpython/cpython/Lib/tkinter/test/test_ttk/test_extensions.py", line 147 in test_variable_change

Again, this was on the Github Ubuntu test (https://github.com/python/cpython/pull/23200/checks?check_run_id=1370499785#step:10:912). I will update my PR to skip *all* tests that use `wait_visbility` on Ubuntu and ask that this would be committed sooner rather than later.
msg381364 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-11-18 17:46
bpo-42400 has been marked as a duplicate of this issue.
msg381366 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-18 17:55
Could that bot be configured to run tests with -u-gui? Tests on Linux I can run manually.
msg381371 - (view) Author: E. Paine (epaine) * Date: 2020-11-18 19:25
> Could that bot be configured to run tests with -u-gui?

This has been found to not just affect one machine: it is both Azure and Github Ubuntu. Therefore, the only Ubuntu bot still running gui tests would be Travis.

It could be done (with a very small patch) but:
1. I not sure about disabling all tkinter gui tests when most appear to be fine still
2. Would this not also stop a number of IDLE tests? (Terry?)
3. Would you intend for this to be temporary or more permanent?
msg381387 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-11-19 03:28
> Would this not also stop a number of IDLE tests? (Terry?)

Yes, at least half of them.  It took years to get us to run gui tests with the X simulator mock.  When we first added gui tests for IDLE in 2013, they (and the t/tk gui tests) were only run on Windows.

Please do not undo that.  Better to kill just wait-visibility on Linux.
msg381723 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2020-11-24 13:11
I became aware of this by the patch proposal "Please submit a PR to skip on Ubuntu", which results in the re-introduction of a distro specific API.

I would rather help fixing this issue, instead on relying on some distro behavor.
msg381739 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2020-11-24 13:35
how is this test run on the CI?  at least it succeeds for me when run locally.
msg381749 - (view) Author: E. Paine (epaine) * Date: 2020-11-24 14:22
PR 23156 [Skip select ttk tests when on Ubuntu] proposes Ubuntu-specific test skips and is given as a justification for PR 28468 [Add platform.freedesktop_osrelease]. However, PR 23474 [Fix timeouts in ttk tests] attempts to correctly address the issue, though proving it does this will be hard as the issue is an intermittent failure and I don't believe any of us can reproduce locally (hence why it was initially blamed on Azure).

Serhiy also raised concerns about skipping on Ubuntu (though for slightly different reasons) and I have attempted to explain the reasoning in https://github.com/python/cpython/pull/23156#issuecomment-732056985.

My problem with using PR 23156 as a justification for PR 28468 is that the changes I proposed were only intended to be temporary (hence why I considered it acceptable to use the os.uname hack rather than something more reliable) and I don't think it, in itself, should cause a permanent API addition.
msg381774 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-11-24 20:57
'Intermittent' appears to mean under 1%, but to much to ignore.  Serhiy's patch is based on some definite hypothesis as to the reason.  The test of his patch would be to merge it and see if failure re-occurs or not. Doing so seems better than nothing.  Additional comment on the PR.
msg382119 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2020-11-30 08:24
New changeset 6cc2c419f6cf5ed336609ba01055e77d7c553e6d by Serhiy Storchaka in branch 'master':
bpo-42142: Try to fix timeouts in ttk tests (GH-23474)
https://github.com/python/cpython/commit/6cc2c419f6cf5ed336609ba01055e77d7c553e6d
msg382125 - (view) Author: miss-islington (miss-islington) Date: 2020-11-30 08:43
New changeset a5b0c17e0d04931e639c4aa7a908a69cd0b529b3 by Miss Islington (bot) in branch '3.8':
bpo-42142: Try to fix timeouts in ttk tests (GH-23474)
https://github.com/python/cpython/commit/a5b0c17e0d04931e639c4aa7a908a69cd0b529b3
msg382126 - (view) Author: miss-islington (miss-islington) Date: 2020-11-30 08:44
New changeset 03ae7e4518dae6547576c616173106d2eb283ae2 by Miss Islington (bot) in branch '3.9':
bpo-42142: Try to fix timeouts in ttk tests (GH-23474)
https://github.com/python/cpython/commit/03ae7e4518dae6547576c616173106d2eb283ae2
History
Date User Action Args
2020-11-30 08:44:08miss-islingtonsetmessages: + msg382126
2020-11-30 08:43:06miss-islingtonsetmessages: + msg382125
2020-11-30 08:24:31miss-islingtonsetpull_requests: + pull_request22447
2020-11-30 08:24:24miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request22446
2020-11-30 08:24:21serhiy.storchakasetmessages: + msg382119
2020-11-24 20:57:45terry.reedysetmessages: + msg381774
2020-11-24 14:22:55epainesetmessages: + msg381749
2020-11-24 13:35:46dokosetmessages: + msg381739
2020-11-24 13:11:57dokosetnosy: + doko
messages: + msg381723
2020-11-23 10:19:20serhiy.storchakasetpull_requests: + pull_request22367
2020-11-19 07:56:10vstinnersetnosy: - vstinner
2020-11-19 03:28:39terry.reedysetmessages: + msg381387
2020-11-18 19:25:33epainesetmessages: + msg381371
2020-11-18 17:55:04serhiy.storchakasetmessages: + msg381366
2020-11-18 17:46:41vstinnersetnosy: + vstinner
messages: + msg381364
2020-11-18 17:20:29christian.heimeslinkissue42400 superseder
2020-11-08 15:56:34epainesetmessages: + msg380555
2020-11-06 16:00:34vstinnersettitle: FAIL tkinter ttk LabeledScale test_resize, and more -> test_ttk timeout: FAIL tkinter ttk LabeledScale test_resize, and more
2020-11-05 20:54:12epainesetfiles: + ismapped.py

messages: + msg380433
2020-11-04 23:06:04epainesetkeywords: + patch
stage: patch review
pull_requests: + pull_request22067
2020-11-04 22:52:48terry.reedysetmessages: + msg380377
2020-11-04 22:11:58epainesetmessages: + msg380371
2020-10-27 00:15:35steve.dowersetmessages: + msg379702
2020-10-26 22:29:17terry.reedysetnosy: + steve.dower
messages: + msg379695
2020-10-26 20:47:24epainesetnosy: + epaine
messages: + msg379686
2020-10-25 03:09:28terry.reedysetmessages: + msg379556
components: + Tests, Tkinter
title: FAIL tkinter ttk LabeledScale test_resize -> FAIL tkinter ttk LabeledScale test_resize, and more
2020-10-24 23:29:20terry.reedycreate