test_tk fails on OS X if test is run from a daemon process without the privilege to access the window server, say a buildbot slave without anyone logged in to the console.  The Tk support needs to check whether it has access rights to the window server before trying to access it.

Workaround is to log buildbot into the console of the OS X machine (which usually but not always works), or to run tests manually.

[More info from Ronald Oussoren]

This is a bug in Tk: 

>>> root = Tkinter.Tk()
Thu May 13 20:45:13 Rivendell.local python[84887] <Error>: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
>>> 2010-05-13 20:45:16.751 Python[84887:d07] An uncaught exception was raised
2010-05-13 20:45:16.762 Python[84887:d07] Error (1002) creating CGSWindow
2010-05-13 20:45:16.955 Python[84887:d07] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Error (1002) creating CGSWindow'
*** Call stack at first throw:
(
	0   CoreFoundation                      0x00007fff85e31d24 __exceptionPreprocess + 180
	1   libobjc.A.dylib                     0x00007fff860000f3 objc_exception_throw + 45
	2   CoreFoundation                      0x00007fff85e31b47 +[NSException raise:format:arguments:] + 103
	3   CoreFoundation                      0x00007fff85e31ad4 +[NSException raise:format:] + 148
	4   AppKit                              0x00007fff84614aba _NSCreateWindowWithOpaqueShape2 + 473
	5   AppKit                              0x00007fff845a9055 -[NSWindow _commonAwake] + 1214
	6   AppKit                              0x00007fff845c6d3d -[NSWindow _makeKeyRegardlessOfVisibility] + 96
	7   AppKit                              0x00007fff845c6cb2 -[NSWindow makeKeyAndOrderFront:] + 24
	8   Tk                                  0x000000010075b86c XMapWindow + 155
	9   Tk                                  0x00000001006ca6d0 Tk_MapWindow + 89
	10  Tk                                  0x00000001006d35e6 TkToplevelWindowForCommand + 2658
	11  Tcl                                 0x00000001006300d3 TclServiceIdle + 76
	12  Tcl                                 0x00000001006162ce Tcl_DoOneEvent + 329
	13  _tkinter.so                         0x0000000100595683 Tkapp_CallDeallocArgs + 277
	14  readline.so                         0x00000001001f1f9a initreadline + 1280
	15  Python                              0x00000001000076a1 PyOS_Readline + 239
	16  Python                              0x0000000100008a57 PyTokenizer_FromString + 1322
	17  Python                              0x00000001000090a0 PyTokenizer_Get + 154
	18  Python                              0x0000000100005698 PyParser_AddToken + 1018
	19  Python                              0x00000001000a2320 PyParser_ASTFromFile + 146
	20  Python                              0x00000001000a443f PyRun_InteractiveOneFlags + 345
	21  Python                              0x00000001000a4615 PyRun_InteractiveLoopFlags + 206
	22  Python                              0x00000001000a4685 PyRun_AnyFileExFlags + 76
	23  Python                              0x00000001000b0286 Py_Main + 2718
	24  Python                              0x0000000100000e6c start + 52
	25  ???                                 0x0000000000000001 0x0 + 1
)
terminate called after throwing an instance of 'NSException'
Abort trap


This is running /usr/bin/python in a session as a user that doesn't have access to the GUI.  The text above says that there is an uncaught ObjC exception, caused by the lack of a connection to the window server. Tk should have converted that to its own style of errors but didn't.

It's fairly easy to create a restricted process tree for testing.  ssh into a Mac which has no one logged into the console, from another machine, and use that connection to launch an xterm or Xemacs window to the other machine.  Then log out of the ssh session.  The session running in the xterm will now have no access privileges to the window server, and can be used for testing.

I've filed an issue for this in the Tcl/Tk tracker: <https://sourceforge.net/tracker/?func=detail&aid=3002320&group_id=10894&atid=110894>

(Assuming that this is the canonical tracker for the Tcl/Tk project, it was the first hit in google for 'tk bug tracker').

BTW. Another way of testing, assuming you have two accounts: use 'su - otheraccount' to get a shell session as another user then try to start tk using:

import Tkinter
root = Tkinter.Tk()

This will currently crash the interpreter due to an uncaught Objective-C exception (as seen in msg105735)

The attached patch is a first start at working around the crash. With this patch I can use Tk without a crash:

>>> import Tkinter
>>> Tkinter.Tk()
Sun May 16 12:11:08 Rivendell.local python.exe[55984] <Error>: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
<Tkinter.Tk instance at 0x100592d88>
>>> 2010-05-16 12:11:08.771 python.exe[55984:903] setCanCycle: is deprecated.  Please use setCollectionBehavior instead
Unexpected Objective-C exception in Tk
>>>

This isn't ideal though, especially because the patch requires that you build using an SDK or explicitly specify a deployment target (MACOSX_DEPLOYMENT_TARGET=10.5) during build.

Another reason to dislike this: the errors are swallowed instead of propagated to Python code.

I haven't checked yet if this patch allows you to run the tests without crashing.

Something I forgot to mention: the patch introduces "_tkinter.m" to enable compiling the _tkinter extension as Objective-C code.  The compiler also supports passing "-x objective-c" to compile _tkinter.c in Objective-C mode, but this flag must be specified early on the GCC command-line and I haven't been able to trick distutils in passing the flags at the right place yet.

Just adding ['-x', 'objective-c'] to the extra_compile_args argument to Extension does not work: that appends the flags to the compiler command-line while the -x flag only works when it is before the name of the source file(s).

Tk has a different tracker than tcl itself, the upstream issue is now <https://sourceforge.net/tracker/?func=detail&aid=3002484&group_id=12997&atid=112997>

I just ran into this while trying to run the test suite with -uall while sshed into an OSX machine and running a non-framework build.  This makes it kind of hard to run the full test suite.

Is there some way to detect that we don't have access to the window server and skip the test?  It's going to fail anyway, so even once the crash bug is fixed skipping the test when we don't have access to the window server is the right thing to do.

In python 2.x  'MacOS.WMAvailable()' returns True if the windowserver is available.

The whole MacOS extension is gone in 3.x, although it should be easy enough to reimplement WMAvailable() using ctypes.

Only OSX 10.4.0 I can run test_tk:

rdmurray@buddy:~/python/py3k>./python.exe -m test.regrtest -uall test_tk         
[1/1] test_tk
Thu Oct 21 12:41:16 buddy.home.bitdance.com python.exe[93560] <Error>: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
_RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
1 test OK.

But test_ttk_guionly crashes with the abort trace after printing the above.  I presume this means test_ttk_guionly was factored out at some point after this issue was raised, but it is interesting that test_tk produces the warning about the window connection but only _guionly crashes.

See also issue 5120.

Bump, this failure is still happening on the ppc tiger buildbot periodically.

test_ttk_guionly is still crashing regulary on "PPC Tiger 3.x" buildbot, see issue #5120. Can anyone with a Mac look at this issue?

I think this issue should be considered a test environment error.  Since this buildbot is set up in an environment where it is "running headless", that is to say the tests are run under a username that is not logged in to the window server, we should not be trying to run GUI tests there, in particular Tkinter tests.  Yes, in a better world, Aqua TK on OS X should not crash in this situation but it is not a Python bug that it does and it's a bit of a hassle to workaround the Tk bug by trying to determine whether the interpreter is indeed running under a user id that is currently also logged in as the primary GUI user.  Also note that there has been no update on the status of the Tk bug that Ronald filed a year ago; not surprisingly, it's not considered an important issue by the project and it's likely not that important for them to make the effort to fix it.  In the real world, this situation does not happen; you would only be running Tkinter stuff if you are logged in as the current GUI user and then the test does not crash.  So, for this and any other OS X buildbot "running headless", the buildbot configuration should be changed to skip GUI tests:

   make buildbottest TESTOPTS='-uall,-gui'

Can someone with access to the buildbot configuration make that change and ensure the crash goes away?

> Since this buildbot is set up in an environment where it is
> "running headless", that is to say the tests are run under
> a username that is not logged in to the window server, we should
> not be trying to run GUI tests there, in particular Tkinter tests.

If I run test_ttk_guionly in SSH, the process abort. gdb trace:
-----------------------
...
kCGErrorRangeCheck : Window Server communications from outside of session allowed for root and console user only
INIT_Processeses(), could not establish the default connection to the WindowServer.
Program received signal SIGABRT, Aborted.
0x9003d66c in kill ()
(gdb) where
#0  0x9003d66c in kill ()
#1  0x9010e8cf in raise ()
#2  0x9010d422 in abort ()
#3  0x917d7dad in RegisterProcess ()
#4  0x917d7b55 in INIT_Processes ()
#5  0x918080b6 in ProcessManagerLazyInitialization ()
#6  0x917d7aa6 in GetCurrentProcess ()
#7  0x92dd5ef9 in GetSystemUIMode ()
#8  0x92dd5e86 in IsMenuBarVisible ()
#9  0x92e2af35 in GetAvailableWindowPositioningBounds ()
#10 0x0b0a7a31 in TkMacOSXDisplayChanged ()
#11 0x0b0a7bee in TkpOpenDisplay ()
#12 0x0b02ace6 in CreateTopLevelWindow ()
#13 0x0b02b0dd in TkCreateMainWindow ()
#14 0x0b035115 in CreateFrame ()
#15 0x0b035544 in TkCreateFrame ()
#16 0x0b02c564 in Initialize ()
#17 0x010d2789 in Tcl_AppInit (interp=0x12c6010) at /Users/vstinner/cpython/Modules/tkappinit.c:106
#18 0x010cc000 in Tkapp_New (screenName=0x0, className=0x10d2e8c "Tk", interactive=0, wantobjects=0, wantTk=1, sync=0, use=0x0) at /Users/vstinner/cpython/Modules/_tkinter.c:724
#19 0x010d1dcd in Tkinter_Create (self=0x1275838, args=0x517038) at /Users/vstinner/cpython/Modules/_tkinter.c:2938
#20 0x00060241 in PyCFunction_Call (func=0x106c338, arg=0x517038, kw=0x0) at Objects/methodobject.c:81
#21 0x0008de54 in call_function (pp_stack=0xbffff164, oparg=0) at Python/ceval.c:3957
#22 0x00087f81 in PyEval_EvalFrameEx (f=0x1255d28, throwflag=0) at Python/ceval.c:2663
#23 0x0008bf7d in PyEval_EvalCodeEx (_co=0x7a7df8, globals=0x705428, locals=0x705428, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3393
#24 0x00079362 in PyEval_EvalCode (co=0x7a7df8, globals=0x705428, locals=0x705428) at Python/ceval.c:767
#25 0x000e1a9f in run_mod (mod=0x18bdba0, filename=0x1caac8 "<string>", globals=0x705428, locals=0x705428, flags=0xbffffa34, arena=0x1101470) at Python/pythonrun.c:1793
#26 0x000e17a4 in PyRun_StringFlags (str=0x125f520 "import _tkinter; c=_tkinter.create(); print(\"a\")\n", start=257, globals=0x705428, locals=0x705428, flags=0xbffffa34) at Python/pythonrun.c:1727
#27 0x000dfe61 in PyRun_SimpleStringFlags (command=0x125f520 "import _tkinter; c=_tkinter.create(); print(\"a\")\n", flags=0xbffffa34) at Python/pythonrun.c:1300
#28 0x0000714c in run_command (command=0x11010e0, cf=0xbffffa34) at Modules/main.c:260
#29 0x00007f4a in Py_Main (argc=3, argv=0x514028) at Modules/main.c:634
#30 0x000024d3 in main (argc=3, argv=0xbffffb6c) at ./Modules/python.c:59
-----------------------

The best would be to fix _tkinter to catch Tk error, but I agree, in a first time we may just try to detect that we cannot create GUI objects. For example, test if the DISPLAY environment variable is present.

"PPC Tiger 3.x": test_ttk_guionly crashs sometimes, the variable is not set.

"AMD64 Leopard 3.x": test_tcl, test_tk, test_ttk_guionly and test_ttk_textonly are skipped; the variable is set (e.g. DISPLAY=/tmp/launch-lDC93s/:0).

"AMD64 Snow Leopard 2 3.x": test_ttk_guionly pass, DISPLAY var is set (e.g. DISPLAY=/tmp/launch-cuyZPv/org.x:0).

Attached requires_tkinter.patch file adds the check for test_tk and test_ttk_guionly.

Victor, I don't understand what your patch is trying to accomplish.  The problem is not that Tkinter isn't built; the problem is simply at execution time.  Yes, you'll see exactly the same behavior if you are logged in via ssh and the usename you are running under is not logged in as the main GUI user.  The solution is to not run Tkinter stuff in that situation.

> Victor, I don't understand what your patch is trying to accomplish.

It skips test_tk and test_ttk_guionly if the DISPLAY environment variable is not set.

> It skips test_tk and test_ttk_guionly if the DISPLAY environment variable is not set.

Whether DISPLAY is set or not isn't relevant.  What's relevant is whether I'm logged in as the GUI user.  In this example, I'm logging in through ssh using the same user name that is currently logged in as the main GUI user.

   $ ssh xxxx
   $ echo $DISPLAY

   $ /usr/local/bin/python3.2 -m test -u gui test_ttk_guionly
   [1/1] test_ttk_guionly
   1 test OK. 

It's really not worth trying to fix the tests.  The buildbot configuration is incorrect.

@pitrou: How can we fix the configuration of the buildbot?

As discussed on IRC, updated patch skipping test_tk and test_ttk_guionly very early if the gui resource is not set.

If I ssh in to a machine, python should not *crash* if I run gui stuff.  The test suite might generate errors rather than skips (although I would argue that it should generate skips), but it should not crash.

On the other hand I agree that fixing the buildbot config is a reasonable interim solution to the buildbot problem.

Hold off on the buildbot changes for the moment.  I have an idea for a possible workaround/solution.

New changeset ea02eca122b5 by Ned Deily in branch '2.7':
Issue #8716: Avoid crashes caused by Aqua Tk on OSX when attempting to run
http://hg.python.org/cpython/rev/ea02eca122b5

New changeset 279488f5a171 by Ned Deily in branch '3.2':
Issue #8716: Avoid crashes caused by Aqua Tk on OSX when attempting to run
http://hg.python.org/cpython/rev/279488f5a171

New changeset 5b5a33196916 by Ned Deily in branch 'default':
Issue #8716: Avoid crashes caused by Aqua Tk on OSX when attempting to run
http://hg.python.org/cpython/rev/5b5a33196916

Let's try this: when running under OS X, the tk and ttk test runners now perform their initial Tk-available sanity check in a subprocess rather than in the main interpreter process.  If the subprocess fails, the tests are skipped.  There are some differences in behavior across the various OS X releases - 10.4 crashes in more cases than 10.5 and 10.6 - but this should cover them all.  I'll leave this open for a while.  Please log any buildbot regressions here.

New changeset 06cb0d602468 by Ned Deily in branch '2.7':
Issue #8716: Fix errors in the non-OS X path of the 27 backport.
http://hg.python.org/cpython/rev/06cb0d602468

> New changeset ea02eca122b5 by Ned Deily in branch '2.7':
> Issue #8716: Avoid crashes caused by Aqua Tk on OSX when attempting to run
> http://hg.python.org/cpython/rev/ea02eca122b5

> New changeset 06cb0d602468 by Ned Deily in branch '2.7':
> Issue #8716: Fix errors in the non-OS X path of the 27 backport.
> http://hg.python.org/cpython/rev/06cb0d602468

Build #200 (revision 6abbc5f68e20eb01094dbbcf486c2ba0e1e4fa77) of "AMD64 Snow Leopard 2 2.7" crashed:

test_ttk_guionly
make: *** [buildbottest] Segmentation fault

http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%202%202.7/builds/200/

runtktests.check_tk_availability() creates a Tkinter.Button() in a subprocess. It should maybe try to create a ttk.Button() for test_ttk_guionly instead of Tkinter.Button().

python2.7 has MacOS.WMAvailable().  When that function returns False the Tkinter tests should be disabled.

The function is not available in Python 3, but is easy enough to implement using ctypes.

That's puzzling. That particular segfault failure is on test_ttk_guionly but test_tk apparently passed earlier in the run and it seems that this buildbot is being run with a window manager connection available (the changes that I added did not raise an exception and the DISPLAY env variable is set).  Further, it's an intermittent segfault.  At the moment, for this buildbot (http://www.python.org/dev/buildbot/all/buildslaves/parc-snowleopard-1), in recent builds only 2.7 builds 202 and 200 have the segfault; 2.7 builds 203 and 201 do not nor do any of the recent 3.2 or 3.x builds.  So, while the fixes I checked in do appear to prevent segfaults in the "headless" operation case (I was able to reproduce and test this on my systems), these two buildbot segfaults appear to have a different root cause.  I am going to temporarily add Ronald's suggested test for 2.7 in hopes of confirming that the window manager connection is indeed not the issue on the buildbot.  I would also be interested in confirmation that what is checked in now prevents the segfaults when running the tests under a headless ssh.

With regard to "untktests.check_tk_availability() creates a Tkinter.Button() in a subprocess. It should maybe try to create a ttk.Button() for test_ttk_guionly instead of Tkinter.Button()": ttk is not necessarily available in older versions of Tk.  The tests are structured to test for Tk availability first and then separately for ttk availability.

New changeset 18ce15f841cf by Ned Deily in branch '2.7':
Issue #8716: Add temporary code for 2.7 to help diagnose buildbot failure.
http://hg.python.org/cpython/rev/18ce15f841cf

New changeset b5accd8adc3e by Ned Deily in branch '2.7':
Issue #8716: Back out temporary changeset 18ce15f841cf
http://hg.python.org/cpython/rev/b5accd8adc3e

New changeset b5ac5e25d506 by Ned Deily in branch '2.7':
Issue #8716: Instead of relying on Aqua Tk exceptions to detect lack of
http://hg.python.org/cpython/rev/b5ac5e25d506

New changeset b58b0c5c7e96 by Ned Deily in branch '3.2':
Issue #8716: Instead of relying on Aqua Tk exceptions to detect lack of
http://hg.python.org/cpython/rev/b58b0c5c7e96

New changeset 2f7e353f9e83 by Ned Deily in branch 'default':
Issue #8716: Instead of relying on Aqua Tk exceptions to detect lack of
http://hg.python.org/cpython/rev/2f7e353f9e83

Despite evidence to the contrary, the temporary MacOS.WMAvailable code showed that there was *not* a window manager connection.  I'm still not sure what the difference in environments is (perhaps it is just due to a different version of Tcl/Tk on the buildbot) nor why the buildbot only fails intermittently.  In any case, I've taken Ronald's suggestion and replaced the Tk-in-a-subprocess sanity check with a ctypes version of what MacOS.WMAvailable did.  Let's see what how well that works.

I haven't seen any failures yet so let's close it.

I just noticed that the Tk tests still get skipped on my machine, and rediscovered this issue. The reason that MacOS.WMAvailable() returns False on the buildbot is because it runs "./python.exe", which is not part of an application bundle and therefore has no GUI access.

To easily reproduce with a framework install:

* Note that MacOS.WMAvailable() returns True when you run
  the python/pythonw when you are logged in to the machine

* Now copy /Library/Framework/Python.frameworks/Versions/Current/Resources/Python.app/Contents/MacOS/Python to "python.exe"

* Run "./python.exe" and note that MacOS.WMAvailable() now returns
  False (and that sys.prefix still refers to the framework)