classification
Title: Tkinter is not thread safe
Type: crash Stage: test needed
Components: Tkinter Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: PythonInTheGrass, amaury.forgeotdarc, asvetlov, belopolsky, cgohlke, loewis, pitrou, terry.reedy
Priority: normal Keywords:

Created on 2011-01-31 15:40 by PythonInTheGrass, last changed 2012-03-22 20:29 by asvetlov.

Files
File name Uploaded Description Edit
TkinterCrash2.py PythonInTheGrass, 2011-01-31 18:29
TkinterCrash3.py belopolsky, 2011-01-31 19:24
TkinterCrash2.py PythonInTheGrass, 2011-02-05 12:36
Messages (15)
msg127604 - (view) Author: Scott M (PythonInTheGrass) Date: 2011-01-31 15:40
The more I look at GUI support in Python, the more I realize that the lack of basic thread safety in GUI support is simply a bug. I know Java's Swing has the same thread limitation, but that doesn't make it right. Xlib is thread safe. The Windows SDK is thread safe. Python is supposed to be the language that's easy to use, and there is nothing easy about teaching new programmers that they have to mess with queues and timers just to get a basic set of displays running, just because when threads are in use.

I'm in the position of teaching folk with little-to-no programming experience, how to script simple applications in Python. The modules they have to use are inherently threaded, and delivery hunks of data from multiple sources to them. The most natural instinct is to put up some graphs and other widgits to display the data, and all of it is completely trivial right up until I have to explain that drawing a line isn't canvas.line(from, to), but becomes an exercise in Queue.Queue and theRoot.after(n, myself), before you even get to learn about widgits. Threading is supposed to simplify problems, not add to them. Having to hack around with special timers and polling, just to get some simple graphs up, is plain unpythonic.

Please consider this a bug, a glaring misfeature, in a language that is otherwise a very reasonable choice to get technical but non-programmerish people into toolmaking self-sufficiency.
msg127611 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2011-01-31 16:18
One possible solution was presented in issue1252236: move tkinter event loop into Python main loop.  However, to consider this report a bug, we need an example code that shows the behavior that you consider incorrect.  Even then, it is likely that fixing the behavior won't be possible without introducing new features.  I am reclassifying this as a feature request.
msg127638 - (view) Author: Scott M (PythonInTheGrass) Date: 2011-01-31 18:29
I don't have an opinion on 1252236. I'm not certain it would help.

I have an extension that runs a bunch of (alien) threads into Python code. The threads deliver information for all sorts of real world events, asynchronously. Multiple threads are used, because people occasionally want to do blocking operations in these alien threads, but still want to be able to handle other incoming information at full speed, with the other threads.

If any of these threads attempts to update a Tkinter widgit - and this is the first thing I tried to do - then tkinter will except or crash in some horrible way, sooner or later. 

The attached .py (in Python 2.7.1) does it without any extensions. Click on Launch quickly 10-15 times; if it doesn't crash, kill and restart. It rarely takes more than 4-5 runs to get a traceback, or occasionally python itself just crashes. Bottom line, one thread in .mainloop() and another thread calling virtually any Tkinter function, even something as simple as .after(), is an invitation a crash.
msg127645 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2011-01-31 19:11
Hmm, either my hand is too slow or my laptop is too fast, but I cannot reproduce the crash. Can you create a non-interactive script?  Maybe start a separate thread generating "launch" events?

What do you mean by "crash"?  Do you get a python backtrace or OS diagnostic saying that the process was killed by a signal of some sort?  If it is the former, please post the backtrace.  If the latter, can you produce C backtrace?
msg127648 - (view) Author: Alexander Belopolsky (belopolsky) (Python committer) Date: 2011-01-31 19:24
I converted TkinterCrash2.py to 3.x using 2to3 (result attached as TkinterCrash3.py) and it works with 3.2rc2 just fine.
msg127962 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-02-05 01:09
'Thread' is not found in the somewhat skimpy tkinter doc. Given the general state of Python with threads, I think it reasonable to take 'not thread-safe' as the default, making this a feature request. In any case, the point is moot until there is a tested fix. The use case presented, though, does make this look like a desireable new feature.
msg127979 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-05 11:18
My claim is that Tkinter is thread-safe as it stands. A lot of thought has been put into making Tkinter thread-safe, so if there is any claim to the contrary, we would need more details: what exact Python version is being used, what exact operating system is being used, what exact code is run, and what exact output is produced.
msg127981 - (view) Author: Scott M (PythonInTheGrass) Date: 2011-02-05 11:46
I'll look into making the crash easier to reproduce this coming week.

Is Tkinter's thread safety new? Because after I started getting crashes, I did my due diligence in Google and found a number of people writing about how it was necessary to use a Queue and Tkinter's after() timer, to draw from multiple threads. And in experiments in 2.7.1 on Windows, it turned out that it wasn't just the drawing calls from a thread other than the mainloop() thread, that invited a crash; even just trying to set an .after() timer from a different thread caused tracebacks, randomly.

At any rate this is less of an issue now, for me at least. I'm putting together a small graphic library for my coworkers. It will have a tiny fraction of Tkinter's capabilities, it will be Windows-only and won't even have native Windows look and feel; but it will be completely, rock-solid, idiot-proof thread safe - any operation (including widget deletes), any thread, any time, with no polling or timers needed. My coworkers are not technical and will get instantly lost if I have to describe queues and timers just to draw a line on a screen.
msg127982 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-05 12:08
> Is Tkinter's thread safety new?

It's supported on Unix since 1.5.1, and on Windows since 2.3.
msg127985 - (view) Author: Scott M (PythonInTheGrass) Date: 2011-02-05 12:36
The new version runs 40 parabolas, then quits. I usually have to run this version 20 times or so to get the crash, so be patient. In general if it's going to crash it does so in the first 6 or so parabolas. Caveat: creates up to 40 threads, so a bit of a CPU pig. You may want to change the 40 in (self.tracks > 40) to 8.

Here's one crash. I got this by double clicking the .py file from Windows explorer, but I can get them with F5 in IDLE too.
---
UpdateStringProc should not be invoked for type cmdName

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
Exception in thread Thread-6:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 530, in __bootstrap_inner
    self.run()
  File "H:\PMT2\MyProjects\TkinterCrash2.py", line 47, in run
    self.deliverToQueue((self.target, z, y))
  File "H:\PMT2\MyProjects\TkinterCrash2.py", line 129, in arrival_122
    new_yz[1])
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 2201, in create_line
    return self._create('line', args, kw)
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 2189, in _create
    *(args + self._options(cnf, kw))))
TclError: bad screen distance "create"
msg127988 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-02-05 13:47
I can't reproduce either, but the latest traceback posted in msg127985 seems to hint at a parameter marshalling problem. "create" is one of the parameters to the called Tk function, but it seemed to be mistaken for another.

Since it's arguably threading-related, saying "I can't reproduce" doesn't seem like a sufficient reason to close the issue. There can be all kinds of influencing factors (OS, CPU, background tasks) and the marshalling code in Modules/_tkinter.c is far from trivial.
msg127991 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-02-05 14:22
Tk produces "bad screen distance" in Tk_GetScreenMM (convert string to screen millimeters) and TkGetDoublePixels (convert string to number of pixels) when strtod fails on the string being passed. It also produces the error in SetPixelAny (convert object to pixel) if the string doesn't start with a double, and SetMMFromAny (convert object to millimeters) if the string either is no double, or not followed by "[cimp]". Finally, TkPixelParseProc returns the error if TkGetDoublePixels returns a negative number.

In relationship to the canvas line command, TkPixelParseProc is used for converting the -width, -activewidth, and -disabledwidth arguments. None of these are used in the test case, though, so it's not clear which pixel parsing fails specifically.
msg127993 - (view) Author: Scott M (PythonInTheGrass) Date: 2011-02-05 14:49
If it helps, over the many iterations of this test code, there have been two kinds of issues:

1. pythonw.exe crashes with the Windows variant of a SEGV. No traceback, just a crash. These are rare.

2. Evidence of confusion over which string the code should be looking at, and it's always down in the create function of Tkinter. Variations of this confusion have been Int() complaining that it can't translate "None", strings like ".667748474.7464" not being recognized as parameter names...  all of it sounds like Tkinter has somehow managed to be looking at the wrong string. Even when the only Tkinter call I do outside the mainloop thread is .after(), the crashes would happen when I went to draw a line.

My (trivial) experience in extending Python makes me wonder if some reference count, somewhere, didn’t get mismanaged. The only times I ever got a hard crash out pythonw.exe in my own projects is when I mishandled a count.

As a side note, if Tkinter is intended to be thread safe, the documentation should say so. Clearly, and in the first paragraph. Once I started having problems, I started Googling, and everything I read lead me to conclude that neither Tkinter nor wx were even intended to be thread safe, so I've started to write my own GUI code. This is a project I might have skipped if it has been clear that Tkinter is at least intended to be thread safe.
msg127995 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-02-05 15:21
The "UpdateStringProc should not be invoked for type cmdName" message (as quoted above in the traceback) apparently can mean that there's a mismanagement of Tcl reference counts.

From http://sourceforge.net/tracker/?func=detail&atid=110894&aid=1326087&group_id=10894:

“This crash (actually, a panic) hints at defective Tcl_Obj
handling - possibly in the core, more likely in an extension
if you're using one. It indicates that Tcl_GetString() or
Tcl_GetStringFromObj() has been called on a cmdNameType
Tcl_Obj that has no string representation, a state that
should never occur.”

Intuitively, cmdNameType seems to refer to createcommand() / deletecommand(). Also, the following code in Tkinter.py looks a bit suspicious:

    def after(self, ms, func=None, *args):
        """Call function once after given time.

        MS specifies the time in milliseconds. FUNC gives the
        function which shall be called. Additional parameters
        are given as parameters to the function call.  Return
        identifier to cancel scheduling with after_cancel."""
        if not func:
            # I'd rather use time.sleep(ms*0.001)
            self.tk.call('after', ms)
        else:
            def callit():
                try:
                    func(*args)
                finally:
                    try:
                        self.deletecommand(name)
                    except TclError:
                        pass
            name = self._register(callit)
            return self.tk.call('after', ms, name)

That is, we call deletecommand() while the command is still being executed.
Perhaps that, together with concurrent memory allocation from another thread (Tcl_Alloc() doesn't use Python's "Tcl lock"), might explain why the cmdName or other things sometimes become corrupted ("Tcl_DeleteCommand deletes a command from a command interpreter. Once the call completes, attempts to invoke cmdName in interp will result in errors").
msg127996 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-02-05 15:28
> As a side note, if Tkinter is intended to be thread safe, the 
> documentation should say so. Clearly, and in the first paragraph.

I'm no Tkinter specialist but, judging by its source code, Tkinter (the Python module) *is* intended to be thread-safe using locking and/or marshalling.

As for wxPython, you are right that it is not thread safe.
History
Date User Action Args
2012-03-22 20:29:29asvetlovsetnosy: + asvetlov
2011-02-05 15:28:33pitrousetnosy: loewis, terry.reedy, amaury.forgeotdarc, belopolsky, pitrou, cgohlke, PythonInTheGrass
messages: + msg127996
2011-02-05 15:21:42pitrousetnosy: loewis, terry.reedy, amaury.forgeotdarc, belopolsky, pitrou, cgohlke, PythonInTheGrass
messages: + msg127995
2011-02-05 14:49:01PythonInTheGrasssetnosy: loewis, terry.reedy, amaury.forgeotdarc, belopolsky, pitrou, cgohlke, PythonInTheGrass
messages: + msg127993
2011-02-05 14:22:24loewissetnosy: loewis, terry.reedy, amaury.forgeotdarc, belopolsky, pitrou, cgohlke, PythonInTheGrass
messages: + msg127991
2011-02-05 13:47:10pitrousetnosy: + pitrou, amaury.forgeotdarc
type: enhancement -> crash
messages: + msg127988
2011-02-05 12:36:40PythonInTheGrasssetfiles: + TkinterCrash2.py

versions: + Python 2.7, - Python 3.3
messages: + msg127985
nosy: loewis, terry.reedy, belopolsky, cgohlke, PythonInTheGrass
2011-02-05 12:08:09loewissetnosy: loewis, terry.reedy, belopolsky, cgohlke, PythonInTheGrass
messages: + msg127982
2011-02-05 11:46:52PythonInTheGrasssetnosy: loewis, terry.reedy, belopolsky, cgohlke, PythonInTheGrass
messages: + msg127981
2011-02-05 11:18:55loewissetnosy: + loewis
messages: + msg127979
2011-02-05 03:17:50cgohlkesetnosy: + cgohlke
2011-02-05 01:09:08terry.reedysetnosy: + terry.reedy
messages: + msg127962
2011-01-31 19:24:09belopolskysetfiles: + TkinterCrash3.py
nosy: belopolsky, PythonInTheGrass
messages: + msg127648
2011-01-31 19:11:21belopolskysetnosy: belopolsky, PythonInTheGrass
messages: + msg127645
2011-01-31 18:48:45brian.curtinsetnosy: belopolsky, PythonInTheGrass
title: Tkinter is not thread safe (and that's... bad) -> Tkinter is not thread safe
2011-01-31 18:29:47PythonInTheGrasssetfiles: + TkinterCrash2.py

title: Tkinter is not thread safe -> Tkinter is not thread safe (and that's... bad)
messages: + msg127638
nosy: belopolsky, PythonInTheGrass
2011-01-31 16:18:48belopolskysetversions: + Python 3.3, - Python 2.7
nosy: + belopolsky

messages: + msg127611

type: behavior -> enhancement
stage: test needed
2011-01-31 16:11:27brian.curtinsettitle: Tkinter is not thread safe. This is a bug. -> Tkinter is not thread safe
2011-01-31 15:40:32PythonInTheGrasscreate