classification
Title: Race conditions in Tkinter with non-threaded Tcl
Type: crash Stage: patch review
Components: Tkinter Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Ivan.Pozdeev, serhiy.storchaka, terry.reedy
Priority: normal Keywords: patch

Created on 2018-04-10 16:41 by Ivan.Pozdeev, last changed 2018-04-15 14:16 by Ivan.Pozdeev.

Files
File name Uploaded Description Edit
TkinterCrash2-2.py Ivan.Pozdeev, 2018-04-10 16:41 Py2 version, crashing
TkinterCrash3-2-2.py Ivan.Pozdeev, 2018-04-10 16:41 Py3 version, working
TkinterHanders.py Ivan.Pozdeev, 2018-04-15 14:16
Pull Requests
URL Status Linked Edit
PR 6444 open Ivan.Pozdeev, 2018-04-10 17:18
Messages (6)
msg315172 - (view) Author: Ivan Pozdeev (Ivan.Pozdeev) * Date: 2018-04-10 16:41
(Marked only 2.7 as affected but this would affect any branch if built with nonthreaded Tcl.)

When running the attached TkinterCrash2-2.py repeatedly with 2.7.14 and 2.7 head, win7 x64, two kinds of errors pop up randomly:

1. Crashes and freezes.
2. Exceptions on the console like:

Exception in thread Thread-14:
Traceback (most recent call last):
  File "C:\Users\Sasha\Documents\cpython\lib\threading.py", line 801, in __boots
trap_inner
    self.run()
  File "../tkt/TkinterCrash2-2.py", line 50, in run
    self.deliverToqueue((self.target, z, y))
  File "../tkt/TkinterCrash2-2.py", line 133, in arrival_122
    new_yz[1])
  File "C:\Users\Sasha\Documents\cpython\lib\lib-tk\Tkinter.py", line 2328, in create_line
    return self._create('line', args, kw)
  File "C:\Users\Sasha\Documents\cpython\lib\lib-tk\Tkinter.py", line 2310, in _create
    *(args + self._options(cnf, kw))))
ValueError: invalid literal for int() with base 10: 'None'


Running the same code with the minimal required changes (attached as TkinterCrash3-2-2.py) under 3.6 (same platform) goes without any errors.


Diagnostics showed:

1. Under debug Python, a crash became an MSVC double-free assertion error. The stacktrace is: Tkapp_Call->Tkapp_CallDeallocArgs->Tcl_DecrRefCount->TclFreeObj->free

2. The exceptions are caused by a `<canvas> create line` Tk call randomly returning "None" (a string) instead of an integer -- which it should never do according to its doc. Since it happens inconsistently, this also suggests a race condition.

3. In Tkapp_Call and SetVar, Tcl lock isn't held when creating/destroying Tcl objects for the call. The underlying fns modify the global state (free objects list, reference counters), so it should be held.
These two are the only such places in the code.


I've fixed this, will file the PR shortly. Holding the lock when calling Tkapp_CallDeallocArgs in Tkapp_Call eliminated the crashes and freezes, locking objects' creation in the same fn decreased the number of exceptions greatly, and locking creation in SetVar, too, eliminated them completely.


I did not check if objects created with AsObj in SetVar need to be disposed of like in Tkapp_Call.

Also have no idea how to autotest the fix. If someone does, I'm all ears.
msg315271 - (view) Author: Ivan Pozdeev (Ivan.Pozdeev) * Date: 2018-04-13 22:46
My best idea for a test as of now is to run the example ~20 times in a loop (or continuously for a comparable amount of time & threads) and catch and register any exceptions in all threads as per https://stackoverflow.com/questions/18349614/check-if-a-python-thread-threw-an-exception .
msg315283 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-04-14 03:08
I believe I have seen this exact error message within the last few months, and I found one example on SO. https://stackoverflow.com/questions/21209124/weird-error-that-comes-up-during-python-program-run
msg315285 - (view) Author: Ivan Pozdeev (Ivan.Pozdeev) * Date: 2018-04-14 08:56
@terry.reedy Apparently, this bug has gone unnoticed for years (to be precise, since https://github.com/python/cpython/commit/b5bfb9f38c786c3330b2d52d93b664588c42283d in 2002),
and there are all kinds of weird rumors circulating about whether  tkinter is thread-safe or not, and what can be run where ( see e.g. https://stackoverflow.com/search?q=tkinter+thread+safe ).
Having run into it in my project, I set out to get to the bottom of this ( https://github.com/RedFantom/mtTkinter/issues/5 ) ...and looks like I did.
msg315286 - (view) Author: Ivan Pozdeev (Ivan.Pozdeev) * Date: 2018-04-14 09:00
Wait a second... I think I noticed another similar bug.

In Tkapp_CallProc. Same case, Tkapp_CallArgs and Tkapp_CallDeallocArgs are called when not holding both locks.

The attached example doesn't use Python event handlers, so it didn't manifest itself in my tests.
msg315304 - (view) Author: Ivan Pozdeev (Ivan.Pozdeev) * Date: 2018-04-14 20:31
Attached reproducing example for event handlers. If launching more than one EventThread, it abort()'s immediately.
History
Date User Action Args
2018-04-15 14:16:34Ivan.Pozdeevsetfiles: - TkinterHanders.py
2018-04-15 14:16:23Ivan.Pozdeevsetfiles: + TkinterHanders.py
2018-04-14 20:31:10Ivan.Pozdeevsetfiles: + TkinterHanders.py

messages: + msg315304
2018-04-14 09:00:12Ivan.Pozdeevsetmessages: + msg315286
2018-04-14 08:56:26Ivan.Pozdeevsetmessages: + msg315285
2018-04-14 03:08:24terry.reedysetnosy: + terry.reedy
messages: + msg315283
2018-04-13 22:46:30Ivan.Pozdeevsetmessages: + msg315271
2018-04-11 07:26:50serhiy.storchakasetnosy: + serhiy.storchaka

versions: + Python 3.6, Python 3.7, Python 3.8
2018-04-10 17:18:42Ivan.Pozdeevsetkeywords: + patch
stage: patch review
pull_requests: + pull_request6139
2018-04-10 16:41:34Ivan.Pozdeevsetfiles: + TkinterCrash3-2-2.py
2018-04-10 16:41:09Ivan.Pozdeevcreate