Using tkinter.filedialog crashes on macOS 12.0 Beta (21A5294g) on M1 when the open file dialog window is created. Full crash below:

2021-08-04 07:19:04.239 Python[40251:323363] *** Assertion failure in -[NSOpenPanel beginServicePanel:asyncExHandler:], NSVBOpenAndSavePanels.m:1910
2021-08-04 07:19:04.241 Python[40251:323363] -[NSSavePanel beginWithCompletionHandler:]_block_invoke caught non-fatal NSInternalInconsistencyException '<NSOpenPanel: 0x1206062d0> is attempting to advance this Open/Save panel to run phase while another self.advanceToRunPhaseCompletionHandler is in waiting for a previous attempt. An Open/Save panel cannot start to advance more than once.' with user dictionary {
NSAssertFile = "NSVBOpenAndSavePanels.m";
NSAssertLine = 1910;
} and backtrace (
0 CoreFoundation 0x00000001a9d47150 __exceptionPreprocess + 240
1 libobjc.A.dylib 0x00000001a9a986e8 objc_exception_throw + 60
2 Foundation 0x00000001aac3b4a4 -[NSCalendarDate initWithCoder:] + 0
3 AppKit 0x00000001ad1f02b0 -[NSSavePanel beginServicePanel:asyncExHandler:] + 512
4 AppKit 0x00000001ad1f1708 -[NSSavePanel runModal] + 332
5 libtk8.6.dylib 0x00000001013d8c18 showOpenSavePanel + 360
6 libtk8.6.dylib 0x00000001013d99e4 Tk_ChooseDirectoryObjCmd + 992
7 libtcl8.6.dylib 0x00000001011cbafc TclNRRunCallbacks + 80
8 _tkinter.cpython-39-darwin.so 0x0000000100c111a4 Tkapp_Call + 400
9 Python 0x0000000100d66a40 cfunction_call + 96
10 Python 0x0000000100d184e0 _PyObject_Call + 128
11 Python 0x0000000100e10150 _PyEval_EvalFrameDefault + 40288
12 Python 0x0000000100e053f0 _PyEval_EvalCode + 444
13 Python 0x0000000100d1877c _PyFunction_Vectorcall + 364
14 Python 0x0000000100e12590 call_function + 128
15 Python 0x0000000100e0ff08 _PyEval_EvalFrameDefault + 39704
16 Python 0x0000000100e053f0 _PyEval_EvalCode + 444
17 Python 0x0000000100d1877c _PyFunction_Vectorcall + 364
18 Python 0x0000000100e12590 call_function + 128
19 Python 0x0000000100e0ff84 _PyEval_EvalFrameDefault + 39828
20 Python 0x0000000100e053f0 _PyEval_EvalCode + 444
21 Python 0x0000000100e5cce4 run_eval_code_obj + 136
22 Python 0x0000000100e5cbf8 run_mod + 112
23 Python 0x0000000100e5a434 pyrun_file + 168
24 Python 0x0000000100e59d58 pyrun_simple_file + 276
25 Python 0x0000000100e59c04 PyRun_SimpleFileExFlags + 80
26 Python 0x0000000100e79d2c pymain_run_file + 320
27 Python 0x0000000100e7947c Py_RunMain + 916
28 Python 0x0000000100e7a6c4 pymain_main + 36
29 Python 0x0000000100e7a93c Py_BytesMain + 40
30 dyld 0x00000001007990fc start + 520
)
.

Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac?

Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)

Thanks for the report. I verified that the crash also occurs on an Intel Mac VM with Monterey beta 4 using the current python.org universal2 installers for 3.9.6 and 3.10.0rc1; they both have a private copy of Tk 8.6.11; so the problem does not appear to be Apple Silicon (M1) related. For what it's worth, the legacy python.org 3.9.6 Intel-only installer, which uses a copy of Tk 8.6.8, does not crash the Monterey beta 4. And no such failures are observed using any of those installers on the current Big Sur 11.5.1 release.

I used brew.sh to install Python, which I think uses the universal installer (although I can’t check, reinstalling macOS after hours of failing to downgrade back to Big Sur). I can’t remember the exact error, but it seemed to be from Apple’s end since it complained about not having access to the save/open dialog box. After I reinstall macOS, if you need further testing, i’m open to trying.
On Aug 4, 2021, 2:53 PM -0400, E. Paine <report@bugs.python.org>, wrote:

E. Paine <xepaine13@gmail.com> added the comment:

Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac?

Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)

----------
components: +macOS
nosy: +epaine, ned.deily, ronaldoussoren, serhiy.storchaka, wordtech

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue44828\>

---

I used brew.sh to install Python, which I think uses the universal installer

As far as I know, Homebrew builds its own Python and Tk.

After I reinstall macOS, if you need further testing, i’m open to trying.

Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python. It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future. Perhaps someone can follow up directly with the Tk folks.

I built Tcl and Tk 8.6 on Monterey beta (21A5294g) and I see this
traceback in the Wish file dialog demo. Note that this is *not* an
error. The file dialog works fine. This is a non-fatal
NSInternalInconsistencyException which prints a traceback to stderr.
It occurs within a call to [NSSavePanel runModal] which is entirely
Apple code. The runModal method is not deprecated. Most likely, when
Apple releases Monterey the assert command will have been removed,
possibly replaced by a call to NSLog which writes information into a
log file rather than print it to stderr. (NSLog itself prints to
stderr when debugging is enabled.) Of course macOS apps run without
stderr, so you will only see this traceback when running a python
program from the terminal. (Any linux user who runs linux apps from
the terminal is used to seeing lots of stderr messages in stderr. Try
running okular from a terminal sometime. So we shouldn't complain too
much.)

Anyway, this looks harmless and the messages will almost surely go
away in the production version. However, there are several new
deprecations related to specifying preferred file types in a file
dialog which I will address in Tk before Monterey is released.

Beta releases are betas, even for macOS.

• Marc

On Thu, Aug 5, 2021 at 1:23 PM Ned Deily <report@bugs.python.org> wrote:

Ned Deily <nad@python.org> added the comment:

> I used brew.sh to install Python, which I think uses the universal installer

As far as I know, Homebrew builds its own Python and Tk.

> After I reinstall macOS, if you need further testing, i’m open to trying.

Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python. It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future. Perhaps someone can follow up directly with the Tk folks.

----------
nosy: +culler

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue44828\>

---

I should have mentioned that I tested on Intel hardware, not M1. I do not have access to an M1 Apple at this time.

Thanks for looking into this, Marc.

I am also experiencing this issue.

I have an M1 Macbook Air running MaOS Monterey Version 12.0 Beta (21A5304g) with a fresh install of Python 3.9.6.

When attempting to save or open anything from either the editor or console window I get the following error message "The Save file operation failed. The save file operation failed to connect to the open and save panel service.”

I have granted the IDEL full drive access which hasn't resolved the issue. The dialog prevents any savings from occurring making the IDEL useless.

I have submitted this issue through Apple's Feedback tool as well.

Can also confirm this issue, tested on M1 and Intel Mac and both has the same error when using Monterey 12.0 Beta 21A5506j.

On the same computer, using Big Sur, saves the file successfully and no error present. Seems like a Monterey issue, not dependent on specific hardware. (Intel/M1)

Tested on fresh Python 3.9.6 (tk version 8.6) Installation on Big Sur and Monterey Beta (Build 21A5506j) on a M1 Macbook Pro and Early 2015 13" Macbook Pro.

Same issue. Running Monterey beta. Experiencing crash.

Is there any plan for a fix or is the plan to wait for Monterey to release?

Unfortunately, I am still seeing this failure in Monterey beta 9. However, we are no longer alone. Here is a report of the same issue in Android Studio:

https://stackoverflow.com/questions/69068842/android-studio-open-file-operation-failed-the-open-file-operation-failed-to-co

This is reportedly fixed in the Adroid Studio beta, so maybe someone will be able to figure out what Google did to workaround the problem.

I was able to fix this problem for Tk on Monterey beta [21A5543b].
The fix has been committed to the tip of the core-8-6-branch in the
Tk fossil repository.

Here is a synopsis. Tk used to open the file dialog by calling
[NSApp runModalForWindow:panel]. Starting with the release of 10.14
this call would produce a warning message on stderr saying that
[NSOpenPanel runModal] should be used instead. But on systems older
than 10.14, the runModal method would fail because the completion handler
would not get called. Now, with 12.0 (at least in the beta) calling
[NSOpenPanel runModal] produces an error dialog saying "The open file
operation failed to connect to the open and save panel service" and
this would be accompanied by a traceback printed on stderr for an
assertion error. (It was not a "crash" but the file selection would
fail.) However, it turns out that calling [NSApp runModalForWindow:panel]
no longer produces the warning in 12.0, and it works correctly.

So my workaround was to add conditional code that only calls the runModal
method on 10.14 and 10.15 and calls runModalForWindow on other versions
of macOS. I tested on 10.15 and it works there as well.

Needless to say, none of these changes are documented by Apple.

Marc, thanks for the update.

Will there be a Tcl/Tk release soonish that includes this fix?

Alternatively, is there patch that we can apply to the latest release when building the copy of Tk included with the installers?

Thanks for the heads-up about the "10.16" issue, Marc. Sorry I missed that aspect when reviewing your fix as we had run into that "feature" before. I guess the idea was that it made it easier during the early days of Big Sur to build some products that were not expecting the version numbering change. In any case, it isn't an issue for the python.org macOS installer builds; we currently support two variants: universal2 which builds on 11 Big Sur (or later) with Tk 8.6.11 (or later); and the deprecated Intel-only variant which builds on 10.9 with Tk 8.6.8. After all the testing we've done, I'd prefer to not disturb the patches in place for the imminent release of 3.9.8 and a patched 3.10.0 installer; we will pick up the final versions when 8.6.12 releases. I am planning to use 8.6.12rc1 with the also imminent 3.11.0a2 to give it a little exposure but the 10.16 fix is also not an issue for it.

As of 2021-11-02, the macOS installer file on python.org for the 3.10.0 release has been updated to include an updated Tk library that includes this fix. All other files installed by the installer are the same as in the original 3.10.0 installer, other than the Tk libraries and the installer ReadMe file. This updated installer pkg, at https://www.python.org/ftp/python/3.10.0/python-3.10.0post1-macos11.pkg, is now the default download for macOS from python.org and appears on the 3.10.0 release page (https://www.python.org/downloads/release/python-3100/). If you have already installed the original 3.10.0 universal2 installer pkg from python.org and are encountering this problem with file dialog windows on macOS 12 Monterey, you can download and run the updated installer pkg to obtain the updated Tk libraries.

The updated Tk will also be included in the macOS installers provided with the next Python releases: 3.9.8 (expected to be released this week) and 3.10.1 (planned for 2021-12-06), as well as the 3.11.0a2 development preview.

Alas, the celebration was premature. Shortly after posting msg405488 above, I installed the updated 3.10.0 on yet another macOS system and, on this one, I observed a problem: under some circumstances at least, the Save dialog window becomes orphaned and does not disappear until IDLE terminates. Here is one sequence of commands I've used:

• launch IDLE -> IDLE opens an IDLE shell window
• press CMD-N to open a new IDLE edit window
• type Python text into the window with a syntax error:
  x = $1
  print(x)
• press CMD-S to save the text into a file -> file Save window opens
• enter a file name and click on Save -> file is saved and IDLE edit window regains focus
• press F5 to run the script -> "Invalid syntax" popup appears
• click on OK to dismiss popup -> the popup disappears and the edit window regains focus. BUT sometimes (and not always) the Save window reappears behind the edit window and does not respond to Cancel or Save buttons, remaining orphaned until IDLE is exited.

This happens on both Monterey and Big Sur, I haven't tried earlier systems. So, while the current fix does make IDLE file dialogs usable on Monterey, it introduces an orphaned (though seemingly harmless) window on presumably all systems.

The big question is why didn't we see this during earlier testing? In my case, the answer is that on all the systems I tested with, the Save dialog window was in its default "compact" form and thus was completely hidden by the IDLE windows. Plus it doesn't seem to always happen if you vary the sequence of events slightly, not surprisingly. I only noticed the orphaned Save window now because on this latest system the Save window defaulted to the fully expanded version (with the side bar etc) and that Save window was wide enough to not be fully hidden behind the other IDLE windows. Sigh!

Until we can resolve this, I have reverted the changes to the website so that the default macOS download is again the original 3.10.0 installer. The patched installer is still downloadable for testing at https://www.python.org/ftp/python/3.10.0/python-3.10.0post1-macos11.pkg.

I found the cause of the zombie dialog window. I was supposed to call
[parent endSheet] after calling [parent beginSheet]. Adding that stops
the window from reappearing. But it does not solve the whole problem.
Subsequent Command-S presses do not cause the file dialog to reappear.
In fact, they do not even generate a call to showOpenSavePanel, which
is supposed to present the dialog, and is also the function where all
of the changes have been made. So this would seem to be a different bug.

I see that if I open a new editor with Command-N and enter some text
then Command-S works for the new editor.

Subsequent Command-S presses do not cause the file dialog to reappear.

Do you mean Command-S after modifying an IDLE edit window that has already been Save (or was Opened, not New)? In that case, the file dialog shouldn't appear: it saves to the existing file. To change the file, you'd need to do a Save As ... (Shift-Command-S). Or perhaps I'm misunderstanding?

Yes that is what I meant. So I guess everything may be OK now. I did
verify that the Tk demo can repeatedly open file save dialogs.

When you press Command-S on a non-new edit window you do see the flash
of the Save menu, so it would appear that a dialog was meant to appear,
and there is no indication of why it did not. But that is an IDLE UX
issue and in any case it sounds like Tk is not doing anything wrong.

When you press Command-S on a non-new edit window you do see the flash
of the Save menu, so it would appear that a dialog was meant to appear,
and there is no indication of why it did not.

Yes, that is long-standing behavior of IDLE on macOS; I just verified that the Save menu flash also happens with the legacy 8.6.8 Tk. I have never noticed it as a problem before and I'm not aware of any user reports of it so I think we can live with that :)

OK. Shift-Command-S works. I will attach the fossil diff which
should have enough context to indicate where to add the one new line.
Then I will merge this change into the 8.6.12 release candidate

New changeset 4a8b405 by Ned Deily in branch 'main':
bpo-44828: macOS installer: avoid leaving a zombie Save panel in Tk 8.6.12rc1 (GH-29367)
4a8b405

A (hypothetical) explanation: I think that each NSWindow maintains a queue of attached sheets. Failing to call the [parent endSheet] method left the file dialogue NSPanel in the queue, although it had been ordered offscreen and so was not visible. Opening the NSAlert added it to the top of the queue. Closing the alert caused it to be removed from the queue. But, since the queue was not empty, the window manager was supposed to orderFront the next sheet in the queue. That sheet was the file dialog, which had not been destroyed since adding it to the queue had incremented its reference count, and failing to remove it had not decremented its reference count. But the separate process that handles events for the file dialog had exited long ago. So that made the newly re-opened file dialog NSPanel into a zombie.

New changeset 6681a77 by Ned Deily in branch '3.10':
bpo-44828: Avoid leaving a zombie Save panel. (GH-29369)
6681a77

New changeset d53d9e7 by Ned Deily in branch '3.9':
bpo-44828: Avoid leaving a zombie Save panel. (GH-29371)
d53d9e7

OK, thanks to Marc's quick response, it looks like we have conquered the elusive zombie dialog window. So let's try again. Note that the download file name has changed to avoid any confusion:

As of 2021-11-03, the macOS installer file on python.org for the 3.10.0 release has been updated to include an updated Tk library that includes this fix. All other files installed by the installer are the same as in the original 3.10.0 installer, other than the Tk libraries and the installer ReadMe file. This updated installer pkg, at https://www.python.org/ftp/python/3.10.0/python-3.10.0post2-macos11.pkg, is now the default download for macOS from python.org and appears on the 3.10.0 release page (https://www.python.org/downloads/release/python-3100/). If you have already installed the original 3.10.0 universal2 installer pkg from python.org and are encountering this problem with file dialog windows on macOS 12 Monterey, you can download and run the updated installer pkg to obtain the updated Tk libraries.

The updated Tk will also be included in the macOS installers provided with the next Python releases: 3.9.8 (expected to be released this week) and 3.10.1 (planned for 2021-12-06), as well as the 3.11.0a2 development preview.

Could this be bumped to a version update to like 3.10.1 or just replace the old package with this updated one? The package name and format now break automations that relied on matching version names in the url. This pattern has worked since 2.7 releases. For example:

https://www.python.org/ftp/python/3.10.0/python-3.10.0-macos11.pkg no longer works

https://www.python.org/ftp/python/3.9.7/python-3.9.7-macos11.pkg works

Could this be bumped to a version update to like 3.10.1 or just replace the old package with this updated one?

I'm sorry that this caused problems for you. We rarely update the artifacts for a release but, in this case, 3.10.1 was not scheduled for release until the beginning of December (the normal 2-month release cycle) and there have been no problems identified so far with 3.10.0 that warrant expediting 3.10.1, something that would affect all platforms and downstream users and suppliers. On the other hand, this issue was a critical one but only for IDLE and tkinter users of the python.org macOS installer who have upgraded to macOS 12 Monterey, a very much smaller group and there was no easy workaround for those users. Simply replacing the installer without changing the URL file name, a so-called stealth update, is, in general, a very bad practice; it leads to even more confusion and we also want to make sure that the obsoleted installer is no longer being downloaded. As far as I know, we have never made any guarantees about the format of the URL of our download files; and, in fact, the file name format for the default python.org macOS installers *has* changed over the years as we have introduced new installer variants, stopped producing old ones, and changed the default variant, something that is planned to happen again in the upcoming 3.9.8 release. So the only reliable way to automatically find file names is from one of the python.org web pages, like for the release or for the platform (i.e. https://www.python.org/downloads/macos/). It would be better to fully support a REST API to provide access to release file data. There is an open website issue about that: python/pythondotorg#1352

After I posted the previous reply, I saw that there have been other users affected by this, so I have now also made the updated installer available under the old URL.

Awesome, thanks for the clear update reason. Makes more sense now. I was
able to get our software to patch around.

An API to get the installer urls would be super helpful.

Thanks Ned!

On Wed, Nov 3, 2021 at 4:19 PM Ned Deily <report@bugs.python.org> wrote:

Ned Deily <nad@python.org> added the comment:

After I posted the previous reply, I saw that there have been other users
affected by this, so I have now also made the updated installer available
under the old URL.

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue44828\>

---

Heads up! A strange Apple quirk has been identified which could affect the file dialog behavior if the Tk library is compiled on macOS 10.XX and used on macOS 11 or 12. (I am not sure if this applies here.)

I'm pretty sure that's a documented feature. Any code compiled with a 10.15 or earlier SDK will never see version 11.0 or later when using regular in process APIs (and IIRC that includes usage of @available/__builtin_available)

Where do you think that "feature" is documented?

According to wikipedia, it was only the Big Sur beta that identified
itself as 10.16. (And I observed this with the beta). But the release
and, I think, the later Big Sur betas stopped doing that.

But I did eventually find a page showing a tweet that "documents" this behavior:
https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/

I guess obscure tweets have replaced documentation.

If you use the older methods to detect OSVersion, Monterey will also
identify as 10.16. iirc there’s an environment variable to enable or
disable that.

On Thu, Nov 4, 2021 at 7:54 AM Marc Culler <report@bugs.python.org> wrote:

Marc Culler <marc.culler@gmail.com> added the comment:

According to wikipedia, it was only the Big Sur beta that identified
itself as 10.16. (And I observed this with the beta). But the release
and, I think, the later Big Sur betas stopped doing that.

But I did eventually find a page showing a tweet that "documents" this
behavior:

https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/

I guess obscure tweets have replaced documentation.

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue44828\>

---

New changeset 10b0c67 by Ned Deily in branch '3.8':
bpo-44828: Avoid leaving a zombie Save panel (GH-29372)
10b0c67

Where do you think that "feature" is documented?

I'd have to check, but I didn't learn this from Twitter, although it wouldn't surprise me if I learned this from a WWDC talk.

The beta for Big Sur never identified itself as 10.16 other than through this feature. I guess Apple determined that too many applications only looked at the minor version determine if the current system is new enough.

Applications compiled with/linked against a 11.0 or 12.0 SDK will always just see the real system version.

Note that this also affects programs just opening the SystemVersion.plist file, that will get substituted by an alternative version when the opening process links against an older SDK. Calling sw_vers and parsing the output does return the right version though.

Well, not exactly ...

culler@abner ~ % export SYSTEM_VERSION_COMPAT=1
culler@abner ~ % sw_vers
ProductName: Mac OS X
ProductVersion: 10.16
BuildVersion: 20G224
culler@abner ~ % export SYSTEM_VERSION_COMPAT=0
culler@abner ~ % sw_vers
ProductName: macOS
ProductVersion: 11.6.1
BuildVersion: 20G224

But that would only be helpful to me if the operatingSystemVersion
property of an NSProcessInfo object also depended on that environment
variable. Feel free to check the docs for an answer to that question:

https://developer.apple.com/documentation/foundation/nsprocessinfo/1410906-operatingsystemversion

Thanks everyone, especially Marc! python.org macOS installers for 3.9.8, 3.10.0, and 3.11.0a2 with patched versions of Tk to avoid the filedialog problems on macOS 12 Monterey are now released and available for download. We can now close this issue and move on :)

https://discuss.python.org/t/python-3-9-8-and-3-11-0a2-are-now-available/11763

FYI, it appears that 8.6.11 works ok with MacOS 12.1 (released on 13-Dec-2021). 8.6.12 also works with MacOS 12.1. I have not tested either extensively, however.