Using tkinter.filedialog crashes on macOS 12.0 Beta (21A5294g) on M1 when the open file dialog window is created. Full crash below:

2021-08-04 07:19:04.239 Python[40251:323363] *** Assertion failure in -[NSOpenPanel beginServicePanel:asyncExHandler:], NSVBOpenAndSavePanels.m:1910
2021-08-04 07:19:04.241 Python[40251:323363] -[NSSavePanel beginWithCompletionHandler:]_block_invoke caught non-fatal NSInternalInconsistencyException '<NSOpenPanel: 0x1206062d0> is attempting to advance this Open/Save panel to run phase while another self.advanceToRunPhaseCompletionHandler is in waiting for a previous attempt. An Open/Save panel cannot start to advance more than once.' with user dictionary {
    NSAssertFile = "NSVBOpenAndSavePanels.m";
    NSAssertLine = 1910;
} and backtrace (
	0   CoreFoundation                      0x00000001a9d47150 __exceptionPreprocess + 240
	1   libobjc.A.dylib                     0x00000001a9a986e8 objc_exception_throw + 60
	2   Foundation                          0x00000001aac3b4a4 -[NSCalendarDate initWithCoder:] + 0
	3   AppKit                              0x00000001ad1f02b0 -[NSSavePanel beginServicePanel:asyncExHandler:] + 512
	4   AppKit                              0x00000001ad1f1708 -[NSSavePanel runModal] + 332
	5   libtk8.6.dylib                      0x00000001013d8c18 showOpenSavePanel + 360
	6   libtk8.6.dylib                      0x00000001013d99e4 Tk_ChooseDirectoryObjCmd + 992
	7   libtcl8.6.dylib                     0x00000001011cbafc TclNRRunCallbacks + 80
	8   _tkinter.cpython-39-darwin.so       0x0000000100c111a4 Tkapp_Call + 400
	9   Python                              0x0000000100d66a40 cfunction_call + 96
	10  Python                              0x0000000100d184e0 _PyObject_Call + 128
	11  Python                              0x0000000100e10150 _PyEval_EvalFrameDefault + 40288
	12  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	13  Python                              0x0000000100d1877c _PyFunction_Vectorcall + 364
	14  Python                              0x0000000100e12590 call_function + 128
	15  Python                              0x0000000100e0ff08 _PyEval_EvalFrameDefault + 39704
	16  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	17  Python                              0x0000000100d1877c _PyFunction_Vectorcall + 364
	18  Python                              0x0000000100e12590 call_function + 128
	19  Python                              0x0000000100e0ff84 _PyEval_EvalFrameDefault + 39828
	20  Python                              0x0000000100e053f0 _PyEval_EvalCode + 444
	21  Python                              0x0000000100e5cce4 run_eval_code_obj + 136
	22  Python                              0x0000000100e5cbf8 run_mod + 112
	23  Python                              0x0000000100e5a434 pyrun_file + 168
	24  Python                              0x0000000100e59d58 pyrun_simple_file + 276
	25  Python                              0x0000000100e59c04 PyRun_SimpleFileExFlags + 80
	26  Python                              0x0000000100e79d2c pymain_run_file + 320
	27  Python                              0x0000000100e7947c Py_RunMain + 916
	28  Python                              0x0000000100e7a6c4 pymain_main + 36
	29  Python                              0x0000000100e7a93c Py_BytesMain + 40
	30  dyld                                0x00000001007990fc start + 520
)
.

Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac? 

Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)

Thanks for the report. I verified that the crash also occurs on an Intel Mac VM with Monterey beta 4 using the current python.org universal2 installers for 3.9.6 and 3.10.0rc1; they both have a private copy of Tk 8.6.11; so the problem does not appear to be Apple Silicon (M1) related. For what it's worth, the legacy python.org 3.9.6 Intel-only installer, which uses a copy of Tk 8.6.8, does not crash the Monterey beta 4. And no such failures are observed using any of those installers on the current Big Sur 11.5.1 release.

I used brew.sh to install Python, which I think uses the universal installer (although I can’t check, reinstalling macOS after hours of failing to downgrade back to Big Sur). I can’t remember the exact error, but it seemed to be from Apple’s end since it complained about not having access to the save/open dialog box. After I reinstall macOS, if you need further testing, i’m open to trying.
On Aug 4, 2021, 2:53 PM -0400, E. Paine <report@bugs.python.org>, wrote:
>
> E. Paine <xepaine13@gmail.com> added the comment:
>
> Thanks for reporting issue and for including the backtrace. I presume you used the Universal 2 installer, given that you are running an M1 mac?
>
> Kevin, do you have access to the macOS 12 beta to help test whether this is a Tkinter or Tk bug? (I assume the latter, as it is likely Apple have changed the API again)
>
> ----------
> components: +macOS
> nosy: +epaine, ned.deily, ronaldoussoren, serhiy.storchaka, wordtech
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________

> I used brew.sh to install Python, which I think uses the universal installer

As far as I know, Homebrew builds its own Python and Tk.

> After I reinstall macOS, if you need further testing, i’m open to trying.

Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python.  It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future.  Perhaps someone can follow up directly with the Tk folks.

I built Tcl and Tk 8.6 on Monterey beta (21A5294g) and I see this
traceback in the Wish file dialog demo.  Note that this is *not* an
error.  The file dialog works fine.  This is a non-fatal
NSInternalInconsistencyException which prints a traceback to stderr.
It occurs within a call to [NSSavePanel runModal] which is entirely
Apple code.  The runModal method is not deprecated.  Most likely, when
Apple releases Monterey the assert command will have been removed,
possibly replaced by a call to NSLog which writes information into a
log file rather than print it to stderr.  (NSLog itself prints to
stderr when debugging is enabled.)  Of course macOS apps run without
stderr, so you will only see this traceback when running a python
program from the terminal.  (Any linux user who runs linux apps from
the terminal is used to seeing lots of stderr messages in stderr.  Try
running okular from a terminal sometime.  So we shouldn't complain too
much.)

Anyway, this looks harmless and the messages will almost surely go
away in the production version.  However, there are several new
deprecations related to specifying preferred file types in a file
dialog which I will address in Tk before Monterey is released.

Beta releases are betas, even for macOS.

- Marc

On Thu, Aug 5, 2021 at 1:23 PM Ned Deily <report@bugs.python.org> wrote:
>
>
> Ned Deily <nad@python.org> added the comment:
>
> > I used brew.sh to install Python, which I think uses the universal installer
>
> As far as I know, Homebrew builds its own Python and Tk.
>
> > After I reinstall macOS, if you need further testing, i’m open to trying.
>
> Thanks. I think the next step would be to try to reproduce just using Tcl and Tk as this most likely is an issue in Tk rather than Python.  It's also quite possible that the underlying problem will be fixed in a future Monterey beta. Alas, I don't have time to look further into this in the immediate future.  Perhaps someone can follow up directly with the Tk folks.
>
> ----------
> nosy: +culler
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________

I should have mentioned that I tested on Intel hardware, not M1.  I do not have access to an M1 Apple at this time.

Thanks for looking into this, Marc.

I am also experiencing this issue.

I have an M1 Macbook Air running MaOS Monterey Version 12.0 Beta (21A5304g) with a fresh install of Python 3.9.6. 

When attempting to save or open anything from either the editor or console window I get the following error message "The Save file operation failed. The save file operation failed to connect to the open and save panel service.”

I have granted the IDEL full drive access which hasn't resolved the issue. The dialog prevents any savings from occurring making the IDEL useless. 

I have submitted this issue through Apple's Feedback tool as well.

Can also confirm this issue, tested on M1 and Intel Mac and both has the same error when using Monterey 12.0 Beta 21A5506j. 

On the same computer, using Big Sur, saves the file successfully and no error present. Seems like a Monterey issue, not dependent on specific hardware. (Intel/M1)

Tested on fresh Python 3.9.6 (tk version 8.6) Installation on Big Sur and Monterey Beta (Build 21A5506j) on a M1 Macbook Pro and Early 2015 13" Macbook Pro.

Same issue. Running Monterey beta. Experiencing crash.

Is there any plan for a fix or is the plan to wait for Monterey to release?

Unfortunately, I am still seeing this failure in Monterey beta 9.  However, we are no longer alone.  Here is a report of the same issue in Android Studio:

https://stackoverflow.com/questions/69068842/android-studio-open-file-operation-failed-the-open-file-operation-failed-to-co

This is reportedly fixed in the Adroid Studio beta, so maybe someone will be able to figure out what Google did to workaround the problem.

I was able to fix this problem for Tk on Monterey beta [21A5543b].
The fix has been committed to the tip of the core-8-6-branch in the
Tk fossil repository.

Here is a synopsis.  Tk used to open the file dialog by calling
[NSApp runModalForWindow:panel].  Starting with the release of 10.14
this call would produce a warning message on stderr saying that
[NSOpenPanel runModal] should be used instead.  But on systems older
than 10.14, the runModal method would fail because the completion handler
would not get called.  Now, with 12.0 (at least in the beta) calling
[NSOpenPanel runModal] produces an error dialog saying "The open file
operation failed to connect to the open and save panel service" and
this would be accompanied by a traceback printed on stderr for an
assertion error.  (It was not a "crash" but the file selection would
fail.)  However, it turns out that calling [NSApp runModalForWindow:panel]
no longer produces the warning in 12.0, and it works correctly.

So my workaround was to add conditional code that only calls the runModal
method on 10.14 and 10.15 and calls runModalForWindow on other versions
of macOS.  I tested on 10.15 and it works there as well.

Needless to say, none of these changes are documented by Apple.

Marc, thanks for the update.

Will there be a Tcl/Tk release soonish that includes this fix? 

Alternatively, is there patch that we can apply to the latest release when building the copy of Tk included with the installers?

Hi Ronald,
There is no calendar scheduling for Tk releases.  Don Porter decides when they happen.  But I think we are due for another one soonish.  In case it doesn't happen before the next Python release I will attach the patch file for commit a32262e9.  (Note that this is subject to change - I still need to test on 10.14.) I believe that this patch is completely self-contained and can be applied to the 8.6.11 release.

(Hint: Tk uses fossil as its SCM system.  On a fossil timeline, clicking any two nodes generates their diff.  The Tk timeline is at https://core.tcl-lang.org/tk/timeline)

Please help me.  Am new to Python, and don't know enough to post here, but I will try.  Have written a couple of programs that use tkinter, especially tkinter.filedialog.askopenfilenames, and as everyone else mine has quit working since Monterey. I have a macOS Monterey version beta        ( 21A5543b ).  Am using Python3 v3.10.0.  Have tried using v3.11.0a1, but could not even compile, says that import ( PIL ) does not exist.  Put back v 3.10.0 and am back to no filedialog.  Is Apple attempting to do away with the file API.  Just asking. Thank you.

No, Apple is not going to do away with their NSOpenPanel.  There is always some churn when they release a new OS.  Subtle changes to APIs can occur with no warning and no documentation.  Sometimes they are bugs. Sometimes they disappear when the OS is released.  Sometimes they are permanent.  I would not recommend using a beta version of the OS to develop your tkinter app.

Thank you very
Guy DeStefano

On Mon, Oct 11, 2021 at 2:10 PM Marc Culler <report@bugs.python.org> wrote:

>
> Marc Culler <marc.culler@gmail.com> added the comment:
>
> No, Apple is not going to do away with their NSOpenPanel.  There is always
> some churn when they release a new OS.  Subtle changes to APIs can occur
> with no warning and no documentation.  Sometimes they are bugs. Sometimes
> they disappear when the OS is released.  Sometimes they are permanent.  I
> would not recommend using a beta version of the OS to develop your tkinter
> app.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

Thank you very much for the reply. Sorry for previous text.
Guy DeStefano

On Mon, Oct 11, 2021 at 2:10 PM Marc Culler <report@bugs.python.org> wrote:

>
> Marc Culler <marc.culler@gmail.com> added the comment:
>
> No, Apple is not going to do away with their NSOpenPanel.  There is always
> some churn when they release a new OS.  Subtle changes to APIs can occur
> with no warning and no documentation.  Sometimes they are bugs. Sometimes
> they disappear when the OS is released.  Sometimes they are permanent.  I
> would not recommend using a beta version of the OS to develop your tkinter
> app.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

@Guy, thanks for your interest but in the future please don't use the issue tracker as a help forum. There are lots of places to ask about such matters; https://www.python.org/about/help/ has a good list of resources and, among the Python-specific mailing lists at https://www.python.org/about/help/, there are ones specifically for Python usage on macOS (https://mail.python.org/mailman/listinfo/pythonmac-sig) and Tkinter usage (https://mail.python.org/mailman/listinfo/tkinter-discuss).

I appreciate the information, In the future I will do as is stated.  Thanks
for the reply.
Guy DeStefano

On Mon, Oct 11, 2021 at 2:24 PM Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily <nad@python.org> added the comment:
>
> @Guy, thanks for your interest but in the future please don't use the
> issue tracker as a help forum. There are lots of places to ask about such
> matters; https://www.python.org/about/help/ has a good list of resources
> and, among the Python-specific mailing lists at
> https://www.python.org/about/help/, there are ones specifically for
> Python usage on macOS (
> https://mail.python.org/mailman/listinfo/pythonmac-sig) and Tkinter usage
> (https://mail.python.org/mailman/listinfo/tkinter-discuss).
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

This issue is happening for me in with my installation of python 3.10 running the release version of mac os monterey 12.0.1 running on an intel mac, I noticed this behavior trying to open or save files in IDLE.

Marc, thanks for providing the patch for Tk. There are some issues with it, though, at least when used with IDLE. I built a Python 3.10.0+ universal2 installer, like what we provide on python.org, with just the updated Tk and tested it on macOS 10.9, 10.13, 10.14, 10.15, 11 and 12.

First, and most important, there seems to be a typo in the fix that causes the dialog panel to break on macOS 10.14: 

-	if ([NSApp macOSVersion] > 101400) {
+	if ( osVersion >= 101400 && osVersion < 120000) {

That ">=" should be just an ">" as before, I think. At least, making that change unbreaks 10.14.

Second, while the filedialog panel now works on macOS 12 (Monterey) with the patch, there are some side effects, at least when using CMD-S (Save) in a new IDLE edit window. After pressing CMD-S, some small object appears on the screen very briefly (too quickly for me to recognize what it is) and then the expected Save filedialog panel appears; however, unlike on all the other operating system levels tested, the keyboard focus is not on the filename text field in the panel. So when the user starts typing the file name, nothing happens until they do something, like clicking on the file name field, to manually move the keyboard focus to that field. I think most users would find that confusing. I don't know if there's any way to change that, either in Tk, tkinter, or IDLE. But it does seem to be a regression. The same binaries work fine on all of the other macOS versions I tested.

Any ideas?

(Although it's not an IDLE bug, I've added IDLE to the components list since it is impacted by this issue.)

Thanks, Ned, for finding my mistake which you generously called a typo.
I have fixed the inequality in the Tk fossil repository.  Incidentally, there is now a core-8-6-12-rc branch of Tk, also containing the fix.  So it should not be too long before 8.6.12 is released.

I will look at the code and check whether I see the UFO that you report.  As you know, all that Tk does is to open an NSOpenPanel.  Once that is open it is completely handled by the OS.  Tk sees no events from the modal interaction and does not have any way of controlling any aspect of the dialog.  So the only hope would be that it is possible to set some configuration option before opening the NSOpenPanel which would somehow cause the OS to assign focus to the correct widget.  I will try to investigate what options are available.

Hi Ned,  I think this problem is fixed now in the tip of the Tk macosx_filedialog branch.  I am attaching the tkMacOSXDialog.c file from that branch.  I think you should be able to just replace the version in 8.6.11 and be able to build a working version.  I have tested on macOS 10.14, 10.15, 11 and 12.  Can you please test on your systems?

If it looks OK I will merge the changes into 8.6.12-rc so they will appear in the 8.6.12 release when that happens.  It should be fairly soon.

Thanks for the quick response, Marc. The results of testing were mixed. The good news: the new version did seem to solve the Monterey problem of the "UFO" and the loss of keyboard focus. However, the new version causes new problems on other releases.

- On 10.14 and earlier systems, I observed the following new behavior in IDLE: on doing a Save from an Edit Window (either with CMD-S or click on the Save menu item), the Save file pane appears as expected but, after entering the file name and clicking Save on the pane, the Save file pane is not dismissed although the file is actually saved under the expected file name. Clicking on the pane's Save button again brings up a "file already exists, do you want to overwrite" message and after clicking OK to that, both the "overwrite" pane and the underlying Save pane are dismissed.

- On 10.15 only, I stumbled across this behavior (and verified it did not occur with the unpatched 8.6.11 nor does it seem to occur anywhere but on 10.15): in a new IDLE edit window, type some text, then Save to a file (CMD-S). Now alter the text and then use Save As (CMD-SHIFT-S) to attempt to save to another file name. Upon entering CMD-SHIFT-S (or clicking on the Save As... menu item), Tk crashes. I've attached a few crash dumps.

Thanks for doing all the testing, Ned.  I guess that the path forward is now clear.  I will revert to the 8.6.11 code for 10.15 and earlier and use the new code for 11 and later.  Of course the 8.6.11 code already has two cases, for 10.14 and earlier and 10.15 and later.  So I will have some code cleanup to do.

The UFO, by the way, was a standalone file dialog which was first being rendered as a separate window and then immediately attached to the parent window as a sheet.  This was easier to see on a slower machine, like a VMWare VM which does not have graphics acceleration.

Evidently the big change for 10.15 was that the file dialog was changed so that it became managed by a separate process, rather than just borrowing the current process's event queue.  The reason, supposedly, was to make it so that an app running in a sandbox could still read and write the user's files.  It seems that Apple had a hard time making this work, and the UFO as well as the assertion error suggest to me that they still haven't figured it out.  So I thnk we can expect more trouble with the next OS.  At least we have a year before that adventure starts.

Needless to say, having some documentation from Apple would be much more pleasant than the trial and error method that we are forced to use now.  But we know better than to expect that.

Hmmm, the 10.15 segfault seems to occur when writing the filename into the entry widget on the dialog.  So maybe it is actually an issue with reference counting an NSString.  I will have to look at that more carefully.

Hi Ned.  Here is one more attempt, hopefully the final one.  I tested with IDLE on 10.14 (VM), 10.15(hard), 11(hard), 12(VM).  I used Python 3.10.0.  I replaced libtk8.6.dylib with the Tk lib compiled from the tip of the macosx_filedialog branch.  I could not reproduce any of the issues that you reported.  (But I hope you will retry as well, since just replacing the tkMacOSXDialog.c file in 8.6.11 is not the same as using the soon-to-be 8.6.12 release candidate.)

I will attach the tkMacOSXDialog.c that I used.

> Here is one more attempt, hopefully the final one.

I think we have a winner! The issues I've seen all seem to be resolved in this latest round. Thank you, Marc! For the record, my testing for this has been very rudimentary manual testing so I'm certainly not saying there aren't other problems that we haven't run into yet but I'm reasonably confident that, with this Tk fix applied, IDLE behavior with python.org Pythons should be similar on macOS 12 Monterey to macOS 11 Big Sur.

I'm going to merge this Tk patch into our macOS installer build scripts and plan to release it in 3.9.8 which is scheduled for next week. And I'm working with the rest of the release team on how best to provide this fix for 3.10 since 3.10.1 is not planned for another month. I'll also try to test with the Tk 8.6.12 branch soon.

That is great news!  I will now merge the changes into the Tk core-8-6-branch and core-8-6-12-rc branches.

New changeset be8318be05e1a874215fa75b8845ede74b2c69b6 by Ned Deily in branch 'main':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/be8318be05e1a874215fa75b8845ede74b2c69b6

New changeset 54579087c69f95531cbe7a97401c67f104a3e52f by Miss Islington (bot) in branch '3.10':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/54579087c69f95531cbe7a97401c67f104a3e52f

New changeset 8e5e74e3049875e9d834fe4408263676fe21e890 by Miss Islington (bot) in branch '3.9':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276)
https://github.com/python/cpython/commit/8e5e74e3049875e9d834fe4408263676fe21e890

New changeset f19c1a115f782036edac306de0f3f9968c1e1fd6 by Miss Islington (bot) in branch '3.8':
bpo-44828: Avoid tkinter file dialog failure on macOS 12 Monterey (GH-29276) (GH-29279)
https://github.com/python/cpython/commit/f19c1a115f782036edac306de0f3f9968c1e1fd6

Heads up!  A strange Apple quirk has been identified which could affect the file dialog behavior if the Tk library is compiled on macOS 10.XX and used on macOS 11 or 12.  (I am not sure if this applies here.)

The fix for the broken file dialog was to use different calls to display the dialog, depending on the OS version.

The quirk is that code compiled on 10.XX and run on 11 or 12 will report the host OS version as 10.16 (which does not exist) no matter whether the actual version is 11 or 12.  The simplest workaround is to replace
the condition OSVersion < 110000 by the condition OSVersion < 101600.

Since tests like this occur in many places, a more robust workaround has been implemented in the Tk fossil repository and will appear in 8.6.12.

Thanks for the heads-up about the "10.16" issue, Marc. Sorry I missed that aspect when reviewing your fix as we had run into that "feature" before. I guess the idea was that it made it easier during the early days of Big Sur to build some products that were not expecting the version numbering change. In any case, it isn't an issue for the python.org macOS installer builds; we currently support two variants: universal2 which builds on 11 Big Sur (or later) with Tk 8.6.11 (or later); and the deprecated Intel-only variant which builds on 10.9 with Tk 8.6.8. After all the testing we've done, I'd prefer to not disturb the patches in place for the imminent release of 3.9.8 and a patched 3.10.0 installer; we will pick up the final versions when 8.6.12 releases. I am planning to use 8.6.12rc1 with the also imminent 3.11.0a2 to give it a little exposure but the 10.16 fix is also not an issue for it.

As of 2021-11-02, the macOS installer file on python.org for the 3.10.0 release has been updated to include an updated Tk library that includes this fix. All other files installed by the installer are the same as in the original 3.10.0 installer, other than the Tk libraries and the installer ReadMe file. This updated installer pkg, at https://www.python.org/ftp/python/3.10.0/python-3.10.0post1-macos11.pkg, is now the default download for macOS from python.org and appears on the 3.10.0 release page (https://www.python.org/downloads/release/python-3100/). If you have already installed the original 3.10.0 universal2 installer pkg from python.org and are encountering this problem with file dialog windows on macOS 12 Monterey, you can download and run the updated installer pkg to obtain the updated Tk libraries.

The updated Tk will also be included in the macOS installers provided with the next Python releases: 3.9.8 (expected to be released this week) and 3.10.1 (planned for 2021-12-06), as well as the 3.11.0a2 development preview.

Alas, the celebration was premature. Shortly after posting msg405488 above, I installed the updated 3.10.0 on yet another macOS system and, on this one, I observed a problem: under some circumstances at least, the Save dialog window becomes orphaned and does not disappear until IDLE terminates. Here is one sequence of commands I've used:
- launch IDLE -> IDLE opens an IDLE shell window
- press CMD-N to open a new IDLE edit window
- type Python text into the window with a syntax error:
   x = $1
   print(x)
- press CMD-S to save the text into a file -> file Save window opens
- enter a file name and click on Save -> file is saved and IDLE edit window regains focus
- press F5 to run the script -> "Invalid syntax" popup appears
- click on OK to dismiss popup -> the popup disappears and the edit window regains focus. BUT sometimes (and not always) the Save window reappears behind the edit window and does not respond to Cancel or Save buttons, remaining orphaned until IDLE is exited.

This happens on both Monterey and Big Sur, I haven't tried earlier systems. So, while the current fix does make IDLE file dialogs usable on Monterey, it introduces an orphaned (though seemingly harmless) window on presumably all systems.

The big question is why didn't we see this during earlier testing? In my case, the answer is that on all the systems I tested with, the Save dialog window was in its default "compact" form and thus was completely hidden by the IDLE windows. Plus it doesn't seem to always happen if you vary the sequence of events slightly, not surprisingly. I only noticed the orphaned Save window now because on this latest system the Save window defaulted to the fully expanded version (with the side bar etc) and that Save window was wide enough to not be fully hidden behind the other IDLE windows. Sigh!

Until we can resolve this, I have reverted the changes to the website so that the default macOS download is again the original 3.10.0 installer.  The patched installer is still downloadable for testing at https://www.python.org/ftp/python/3.10.0/python-3.10.0post1-macos11.pkg.

I found the cause of the zombie dialog window.  I was supposed to call
[parent endSheet] after calling [parent beginSheet].  Adding that stops
the window from reappearing.  But it does not solve the whole problem.
Subsequent Command-S presses do not cause the file dialog to reappear.
In fact, they do not even generate a call to showOpenSavePanel, which
is supposed to present the dialog, and is also the function where all
of the changes have been made.  So this would seem to be a different bug.

I see that if I open a new editor with Command-N and enter some text
then Command-S works for the new editor.

> Subsequent Command-S presses do not cause the file dialog to reappear.

Do you mean Command-S after modifying an IDLE edit window that has already been Save (or was Opened, not New)? In that case, the file dialog shouldn't appear: it saves to the existing file. To change the file, you'd need to do a Save As ... (Shift-Command-S). Or perhaps I'm misunderstanding?

Yes that is what I meant.  So I guess everything may be OK now.  I did
verify that the Tk demo can repeatedly open file save dialogs.


When you press Command-S on a non-new edit window you do see the flash
of the Save menu, so it would appear that a dialog was meant to appear,
and there is no indication of why it did not.  But that is an IDLE UX
issue and in any case it sounds like Tk is not doing anything wrong.

> When you press Command-S on a non-new edit window you do see the flash
of the Save menu, so it would appear that a dialog was meant to appear,
and there is no indication of why it did not.

Yes, that is long-standing behavior of IDLE on macOS; I just verified that the Save menu flash also happens with the legacy 8.6.8 Tk. I have never noticed it as a problem before and I'm not aware of any user reports of it so I think we can live with that :)

OK.  Shift-Command-S works.  I will attach the fossil diff which
should have enough context to indicate where to add the one new line.
Then I will merge this change into the 8.6.12 release candidate

New changeset 4a8b4051734fd2ce46e15e6369811132ac3a5697 by Ned Deily in branch 'main':
bpo-44828: macOS installer: avoid leaving a zombie Save panel in Tk 8.6.12rc1 (GH-29367)
https://github.com/python/cpython/commit/4a8b4051734fd2ce46e15e6369811132ac3a5697

A (hypothetical) explanation: I think that each NSWindow maintains a queue of attached sheets.  Failing to call the [parent endSheet] method left the file dialogue NSPanel in the queue, although it had been ordered offscreen and so was not visible. Opening the NSAlert added it to the top of the queue.  Closing the alert caused it to be removed from the queue. But, since the queue was not empty, the window manager was supposed to orderFront the next sheet in the queue. That sheet was the file dialog, which had not been destroyed since adding it to the queue had incremented its reference count, and failing to remove it had not decremented its reference count.  But the separate process that handles events for the file dialog had exited long ago.  So that made the newly re-opened file dialog NSPanel into a zombie.

New changeset 6681a77c52df41636feb213d63ba27a759c7e5f4 by Ned Deily in branch '3.10':
bpo-44828: Avoid leaving a zombie Save panel. (GH-29369)
https://github.com/python/cpython/commit/6681a77c52df41636feb213d63ba27a759c7e5f4

New changeset d53d9e7f4f1656a13b030b17baca743455e511fd by Ned Deily in branch '3.9':
bpo-44828: Avoid leaving a zombie Save panel. (GH-29371)
https://github.com/python/cpython/commit/d53d9e7f4f1656a13b030b17baca743455e511fd

OK, thanks to Marc's quick response, it looks like we have conquered the elusive zombie dialog window. So let's try again. Note that the download file name has changed to avoid any confusion:

As of 2021-11-03, the macOS installer file on python.org for the 3.10.0 release has been updated to include an updated Tk library that includes this fix. All other files installed by the installer are the same as in the original 3.10.0 installer, other than the Tk libraries and the installer ReadMe file. This updated installer pkg, at https://www.python.org/ftp/python/3.10.0/python-3.10.0post2-macos11.pkg, is now the default download for macOS from python.org and appears on the 3.10.0 release page (https://www.python.org/downloads/release/python-3100/). If you have already installed the original 3.10.0 universal2 installer pkg from python.org and are encountering this problem with file dialog windows on macOS 12 Monterey, you can download and run the updated installer pkg to obtain the updated Tk libraries.

The updated Tk will also be included in the macOS installers provided with the next Python releases: 3.9.8 (expected to be released this week) and 3.10.1 (planned for 2021-12-06), as well as the 3.11.0a2 development preview.

Could this be bumped to a version update to like 3.10.1 or just replace the old package with this updated one? The package name and format now break automations that relied on matching version names in the url. This pattern has worked since 2.7 releases. For example: 

https://www.python.org/ftp/python/3.10.0/python-3.10.0-macos11.pkg no longer works

https://www.python.org/ftp/python/3.9.7/python-3.9.7-macos11.pkg works

> Could this be bumped to a version update to like 3.10.1 or just replace the old package with this updated one?

I'm sorry that this caused problems for you. We rarely update the artifacts for a release but, in this case, 3.10.1 was not scheduled for release until the beginning of December (the normal 2-month release cycle) and there have been no problems identified so far with 3.10.0 that warrant expediting 3.10.1, something that would affect all platforms and downstream users and suppliers. On the other hand, this issue was a critical one but only for IDLE and tkinter users of the python.org macOS installer who have upgraded to macOS 12 Monterey, a very much smaller group and there was no easy workaround for those users. Simply replacing the installer without changing the URL file name, a so-called stealth update, is, in general, a very bad practice; it leads to even more confusion and we also want to make sure that the obsoleted installer is no longer being downloaded. As far as I know, we have never made any guarantees about the format of the URL of our download files; and, in fact, the file name format for the default python.org macOS installers *has* changed over the years as we have introduced new installer variants, stopped producing old ones, and changed the default variant, something that is planned to happen again in the upcoming 3.9.8 release. So the only reliable way to automatically find file names is from one of the python.org web pages, like for the release or for the platform (i.e. https://www.python.org/downloads/macos/). It would be better to fully support a REST API to provide access to release file data. There is an open website issue about that: https://github.com/python/pythondotorg/issues/1352

After I posted the previous reply, I saw that there have been other users affected by this, so I have now also made the updated installer available under the old URL.

Awesome, thanks for the clear update reason. Makes more sense now. I was
able to get our software to patch around.

An API to get the installer urls would be super helpful.

Thanks Ned!

On Wed, Nov 3, 2021 at 4:19 PM Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily <nad@python.org> added the comment:
>
> After I posted the previous reply, I saw that there have been other users
> affected by this, so I have now also made the updated installer available
> under the old URL.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

> Heads up!  A strange Apple quirk has been identified which could affect the file dialog behavior if the Tk library is compiled on macOS 10.XX and used on macOS 11 or 12.  (I am not sure if this applies here.)

I'm pretty sure that's a documented feature. Any code compiled with a 10.15 or earlier SDK will never see version 11.0 or later when using regular in process APIs (and IIRC that includes usage of @available/__builtin_available)

Where do you think that "feature" is documented?

According to wikipedia, it was only the Big Sur beta that identified
itself as 10.16.  (And I observed this with the beta).  But the release
and, I think, the later Big Sur betas stopped doing that.

But I did eventually find a page showing a tweet that "documents" this behavior:
https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/

I guess obscure tweets have replaced documentation.

If you use the older methods to detect OSVersion, Monterey will also
identify as 10.16. iirc there’s an environment variable to enable or
disable that.

On Thu, Nov 4, 2021 at 7:54 AM Marc Culler <report@bugs.python.org> wrote:

>
> Marc Culler <marc.culler@gmail.com> added the comment:
>
> According to wikipedia, it was only the Big Sur beta that identified
> itself as 10.16.  (And I observed this with the beta).  But the release
> and, I think, the later Big Sur betas stopped doing that.
>
> But I did eventually find a page showing a tweet that "documents" this
> behavior:
>
> https://eclecticlight.co/2020/07/21/big-sur-is-both-10-16-and-11-0-its-official/
>
> I guess obscure tweets have replaced documentation.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue44828>
> _______________________________________
>

New changeset 10b0c671580a2f8dd013b6345c1dc9789d5bd95c by Ned Deily in branch '3.8':
bpo-44828: Avoid leaving a zombie Save panel (GH-29372)
https://github.com/python/cpython/commit/10b0c671580a2f8dd013b6345c1dc9789d5bd95c

> Where do you think that "feature" is documented?

I'd have to check, but I didn't learn this from Twitter, although it wouldn't surprise me if I learned this from a WWDC talk.

The beta for Big Sur never identified itself as 10.16 other than through this feature.  I guess Apple determined that too many applications only looked at the minor version determine if the current system is new enough. 

Applications compiled with/linked against a 11.0 or 12.0 SDK will always just see the real system version.

Note that this also affects programs just opening the SystemVersion.plist file, that will get substituted by an alternative version when the opening process links against an older SDK.  Calling sw_vers and parsing the output does return the right version though.

Well, not exactly ...

culler@abner ~ % export SYSTEM_VERSION_COMPAT=1
culler@abner ~ % sw_vers
ProductName:	Mac OS X
ProductVersion:	10.16
BuildVersion:	20G224
culler@abner ~ % export SYSTEM_VERSION_COMPAT=0
culler@abner ~ % sw_vers                       
ProductName:	macOS
ProductVersion:	11.6.1
BuildVersion:	20G224

But that would only be helpful to me if the operatingSystemVersion
property of an NSProcessInfo object also depended on that environment
variable.  Feel free to check the docs for an answer to that question:

https://developer.apple.com/documentation/foundation/nsprocessinfo/1410906-operatingsystemversion

Thanks everyone, especially Marc! python.org macOS installers for 3.9.8, 3.10.0, and 3.11.0a2 with patched versions of Tk to avoid the filedialog problems on macOS 12 Monterey are now released and available for download. We can now close this issue and move on :)

https://discuss.python.org/t/python-3-9-8-and-3-11-0a2-are-now-available/11763

FYI, it appears that 8.6.11 works ok with MacOS 12.1 (released on 13-Dec-2021). 8.6.12 also works with MacOS 12.1. I have not tested either extensively, however.