Packaging Tcl and Tk within the python framework is a great idea
and a big improvement.  But the way it is being done is pretty crazy,
and makes it unnecessarily difficult to upgrade the version of Tcl/Tk
that python is using.  That is something which should be easy,
especially since python is still using version 8.6.8 even though 8.6.11
is about to be released and includes the results of a huge effort to
improve and stabilize the macOS port of Tk.

There is a standard and completely straightforward way of including
a framework within a framework.  The normal thing to do would be
to create a directory Python.framework/Versions/X.Y/Frameworks/ which
contains Tcl.framework and Tk.framework.  Upgrading to a newer version
of Tcl and Tk would then simply require updating those frameworks, which
would be very easy to accomplish, for example, with a .pkg file.

Instead of doing this, Python currently jams everything related to Tcl
and Tk into its lib directory, changing the library and directory names
randomly and breaking the standard directory structure that Tcl and Tk
use in their frameworks.

Why not make it standard and simple?  Why add all of this random noise?

It sounds like your solution is macOS only, where as Python needs to be cross platform.

Further, I assume there are good reasons that it's implemented the way it is. Perhaps there was no alternative when it was initially developed? And we're constrained by backward compatibility to at least some extent.

I'm sure that we'd accept contributions that used a more modern approach, if they were cross platform and didn't break anything.

By the way, getting volunteers to pay attention to your issue isn't helped by using words like "pretty crazy" and "random noise".

Dear Eric,

I am aware that python is cross platform.

The change that I am proposing would be almost entirely limited to
the file:
 Mac/BuildScript/build-installer.py
which is entirely mac-specific.  It controls how the Mac installer is
constructed.  Specifically, the effect would be to change the
directory structure of the Python framework which is installed by the
Python installer package in /Library/Frameworks/Python.framework.

There presumably would also be a very small change in the _tkinter
module to account for the fact that the Tcl and Tk shared libraries
would acquire new pathnames.

My sincere apologies for offending you by my use of the words
"crazy" and "random noise". I hope you will be able to focus on the
content of the ticket in spite of my use of those offensive words.

I'm not sure what you are trying to accomplish.  It is already possible to replace the Tcl/Tk libraries by replacing the relevant files in Python.framework. However: these aren't meant to be user serviceable parts, the copy of Tcl/Tk that we include is tested with Tkinter.

I assume a future release of python will include a new copy of Tk when tests indicate it is an improvement for Tkinter. I won't make any promises because I'm not doing the integration work.

Dear Ronald,

You are correct that "It is already possible to replace the Tcl/Tk libraries by replacing the relevant files in Python.framework."
I know that because I have done it.  Moreover, having done it, I
know that it is much more difficult than it should be.  A large
part of the difficulty is that the directory structure of the
Tcl/Tk installation within the Python framework is undocumented and
completely different from that used within Tcl.framework and
Tk.framework.  There is no need for that.  The versions of those
frameworks with Python wants to use could simply be embedded within
the Python.framework as subframeworks.  That simplifies the build and
it simplifies upgrades and it costs nothing as far as Python is
concerned.

The backwards compatibility promise offered by Tk and Tcl means that
replacing Tcl/Tk 8.6.X with Tcl/Tk 8.6.Y will work with no change to
the binary _tkinter.ZZZZ module.  When improvements or bug fixes are
made to Tcl and Tk it should be straightforward for users to update
Python so that it will take advantage of those improvements.  Before
Tcl and Tk were included in the Python framework that was simple.
You could just install the newer release of Tcl/Tk (at least for patch
releases).  Since _tkinter would look in the standard places, i.e.
/Library/Frameworks/Tk.framework (or Tcl.framework) for its shared
libraries, that would automatically work.

The point is that it could, and should, be just as simple to make such
an upgrade with the new where Tcl and Tk are included within the Python.framework.

That is what I am trying to accomplish.  The scope of this change will
probably be clearer if I make a pull request.  So I will do that.

Marc, thanks for your suggestion. I'm the one responsible for how Tcl and Tk are currently packaged in the python.org framework build and I know exactly what you are suggesting so there is no need for you to make a PR. There were reasons why I chose to set things up the way they are now but it doesn't mean we need to continue to do it that way.  I will be looking at macOS packaging issues this week during our development sprint and I will respond further to this issue then.

Ned - do you have any news on the topic of packaging Tcl/Tk within the
Python bundle?

Hi Ned, I have a comment about the code signing issues that would arise if the Tcl and Tk frameworks were embedded as actual subframeworks inside the Python.framework, and a user later replaced those with newer versions. The answer is that no new issues arise.  The current Python.framework fails codesign verification as soon as a single new package is installed with pip (even upgrading pip itself).  The codesign command detects the new files and exits with non-zero status if called with --verify.  In verbose mode it will list all of the files that were added.

This does not prevent python from working.  So it should just be acknowledged that python's codesigning is used purely to make installation simpler and that no attempt is currently being made to ensure that an installed python framework continues to pass codesign verification.  As a consequence, my proposal in this ticket would raise no new codesigning issues, but it would allow users to much more easily upgrade the Tcl and Tk embedded in /Library/Frameworks/Python.framework versions.  It would also simplify the recipe for building Tcl and Tk in the buildscript.

While the python framework may be viewed as a black box, in fact it is not a black box at all.  It can be changed at will either by python itself or by a user.  Making its structure clearer and cleaner could only be an improvement.