This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Tkinter multi-processing performance, Linux 10-25 times faster than Windows 10
Type: performance Stage:
Components: Tkinter, Windows Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gpolo, james.mccormack, josh.r, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2019-03-23 15:57 by james.mccormack, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
mosaic.py james.mccormack, 2019-03-26 10:57
Messages (5)
msg338682 - (view) Author: J.E.McCormack (james.mccormack) * Date: 2019-03-23 15:57
I am measuring multi-process GUI performance (Tkinter 8.6, Python 3.7) for drawing lines, circles, text boxes, etc. In a fairly typical experiment on a i7-6700HQ, 4-core (8 thread), on Windows 10 I measure 25.5k objects/sec for one process running alone, and 19.9k objects/sec total for eight processes. For Linux Kubuntu KDE desktop the figures are 61k objects/sec and 230k objects/sec (a multi-core boost of times 3.8). For running eight processes, the performance difference, KDE vs Win10, is therefore times 11.6. The difference over a range of tests is 10-25 times.
	
Clearly Win10 is not doing multi-core. Perhaps Tkinter is calling a Windows SDK function which is not thread-safe within the Windows GDI, imposing a single-thread barrier system-wide?
	
I am just wondering, firstly, if I have simply missed mention of this limitation anywhere. I can supply more info if needed.
msg338805 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2019-03-25 15:53
Could you provide a minimal reproducer script? With multiprocessing involved, I'd suspect some issue with data sharing (Windows can't fork after all, so it's possible something is involved there).

I see nothing obvious in the _tkinter module that would explain this, which leaves Tk itself or multiprocess communications as the likely cause; a repro script would at least guide investigation.
msg338808 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-03-25 16:14
Windows only allows a single thread to access Win32 GUI elements at a time, and I'm fairly sure whichever part of Tcl/Tk/Tkinter is responsible for this makes sure it happens.

So if you're throwing lots of UI updates at the UI thread, then yeah, you're going to cause massive contention there. Not sure there's any way around it other than "don't do that" - while you *could* lock individual data structures, Windows doesn't permit that because it turns out people get it wrong and make programs that crash.
msg338820 - (view) Author: J.E.McCormack (james.mccormack) * Date: 2019-03-25 19:53
I can run four independent processes (i.e. not using multiprocessing, with no links at all between them) yet the results show that only one core is running. Where is this lock taking place? Why would a tkinter process need to know about another tkinter process?
	
A little bit of history I have learned is that prior to Windows 7, the GDI sub-system imposed a global lock system-wide so that only one process (one thread) could write to the display at one time. This meant in effect it was a one-core GUI desktop. From Windows 7, this was supposed to have been 'fixed', but all I have read is that the "GDI lock became more fine-grained, reducing concurrency bottlenecks". I wonder did anyone ever measure performance in real-world scenarios to demonstrate whether there was in fact any improvement?
msg338871 - (view) Author: J.E.McCormack (james.mccormack) * Date: 2019-03-26 10:57
Attached is a minimal reproducer script which is sufficient to show the issue clearly.
	
It is a simple single-thread Tkinter program with one canvas. No multiprocessing, no shared variables, no connections between instances. Instructions at top of file.
	
Results on i7-6700HQ, 4-core (8 thread), 2.60GHz, 16GB, Windows 10:-
    
    1 process running alone:             29k objects/sec
    6 processes running concurrently:    4.3k objects/sec each, = 25.8k objects/sec combined.

Conclusion: One-core performance, global system-wide lock.
History
Date User Action Args
2022-04-11 14:59:12adminsetgithub: 80589
2019-03-26 10:57:58james.mccormacksetfiles: + mosaic.py

messages: + msg338871
2019-03-25 19:53:58james.mccormacksetmessages: + msg338820
2019-03-25 16:14:38steve.dowersetmessages: + msg338808
2019-03-25 15:53:02josh.rsetnosy: + josh.r
messages: + msg338805
2019-03-25 09:01:07SilentGhostsetnosy: + gpolo, paul.moore, serhiy.storchaka, tim.golden, zach.ware, steve.dower
components: + Windows
2019-03-23 15:57:48james.mccormackcreate