This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python execution broken after update via Windows Store
Type: Stage:
Components: Windows Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, firewave, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-11-11 20:17 by firewave, last changed 2022-04-11 14:59 by admin.

Files
File name Uploaded Description Edit
read_appexec.py eryksun, 2021-11-12 13:23
Messages (19)
msg406176 - (view) Author: (firewave) Date: 2021-11-11 20:17
I have Python 3.9 installed from the Windows Store and use it in CMake scripts.

After the recent update to 3.92288.0 Python executions suddenly started failing with "Access is denied". I assumed I was using a wrong or cached path but that didn't turn out to be the case.

It did find the executable in the (apparently) correct folder "C:/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0/python3.9.exe".

I tried running that via the command-line and it failed with "Access is denied." as well. It appears to be related to "C:/Program Files/WindowsApps" requiring Administrator permissions.

Also just running "python3.9.exe" (which exists in the PATH) no longer works. It fails with "The system cannot find the file C:\Users\<username>\AppData\Local\Microsoft\WindowsApps\python3.9.exe."

So it seems the update messed something up.

I had Python 3.9 installed and working fine for some version now. I ran into something similar in the past and had to uninstall and install it again to fix the problem.

I am using Windows 10 Version 21H1 (OS Build 19043.1348).
msg406177 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-11 20:56
If you open "Manage app execution aliases", do you see the entries for Python? I've heard that they sometimes switch themselves off during updates, but nobody seems to know why. Even if not, disabling and re-enabling them may help.

The executable in Program Files cannot be launched except in a few oddly specific circumstances. I've spoken to the team responsible and they consider this part of the feature, so the execution aliases mentioned above are the way to go (these live in your own AppData folder - check sys.executable once you get it going again).
msg406178 - (view) Author: (firewave) Date: 2021-11-11 21:41
I checked them and they look fine. I have Python 3.10 installed as well so the non-suffix and "3" suffix-only executable refer to that. All the "3.9" suffix ones point to the "Python 3.9" entries and are enabled.
msg406180 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-12 00:01
Can you look in your %LocalAppData%\Microsoft\WindowsApps folder and see 
if the python*.exe files are there? You should be able to double-click 
them to launch them.

That folder should be on your PATH so that they work everywhere, and it 
should reflect what you see in Manage App Execution Aliases. If it 
doesn't, that's a Windows bug, and I'll need as much detail as we can 
get to reproduce and report it to them.
msg406190 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-12 06:04
>  It fails with "The system cannot find the file 
>  C:\Users\<username>\AppData\Local\Microsoft\WindowsApps\python3.9.exe."

The shell's CreateProcessW() call failed with ERROR_FILE_NOT_FOUND (2). The file exists, but it's probably a broken appexec link. I'm attaching a script that displays the contents of an appexec link. For example, here's the working "python3.9.exe" link:

    C:\>read_appexec.py "%LocalAppData%\Microsoft\WindowsApps\python3.9.exe"
    Version: 3
    Package ID: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0
    Entry Point: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0!Python
    App Type: 0
    Target Path:
    C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0\python3.9.exe

---

> The executable in Program Files cannot be launched except in a few
> oddly specific circumstances.

The SYSTEM, LOCAL SERVICE, and NETWORK SERVICE accounts can directly execute the real executable. For standard users, however, a conditional access control entry is set in the file's permissions that allows execute access only if the accessing security context has the app's WIN://SYSAPPID, such as "PYTHONSOFTWAREFOUNDATION.PYTHON.3.9_QBZ5N2KFRA8P0". When CreateProcessW() spawns an app from an appexec link, it adds the required WIN://SYSAPPID to a new token that gets created for the process. The app itself can thus directly load and execute binary images (EXEs, DLLs) from its installation directory under "%ProgramFiles%\WindowsApps".
msg406206 - (view) Author: (firewave) Date: 2021-11-12 12:15
Thanks a lot for all the details.

I used the script to check all the 3.9 appexec links and only the "python3.9" one points to the wrong executable.

Version: 3
Package ID: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0
Entry Point: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0!Python
App Type: 0
Target Path:
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64__qbz5n2kfra8p0\python3.9.exe

Version: 3
Package ID: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0
Entry Point: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0!Pip
App Type: 0
Target Path:
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0\pip3.9.exe

Could it be possible that it might not be able to replace that link when it is currently in use?

Also the "Reliability Monitor" shows a warning to install the "Python 3.9" app but I never saw anything about it in the "Windows Store".

Installation Failure: Windows failed to install the following update with error 0x80073D02: 9P7QFQMJRFP7-PythonSoftwareFoundation.Python.3.9.

A day later the success entry is found.

Installation Successful: Windows successfully installed the following update: 9P7QFQMJRFP7-PythonSoftwareFoundation.Python.3.9
msg406211 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-12 13:23
> Could it be possible that it might not be able to replace that 
> link when it is currently in use?

Appexec links are only used briefly to find the real executable and create the access token that allows executing it. I don't know how your local profile directory ended up with a stale link. Maybe Steve has more insight.

There should be a "PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0" subdirectory in your local "WindowsApps" directory. Check the "python3.9.exe" link in that directory as well. It's the basis for the link of the same name that gets created in the parent directory.

Note that I wrote read_appexec.py in a hurry based on someone's blog (appexec links are undocumented), which I shouldn't have trusted. I've uploaded a new version just now with what are hopefully more accurate fields and field names. The previous target path result was correct, however. For example:

    C:\>read_appexec.py "%LocalAppData%\Microsoft\WindowsApps\python3.9.exe"
    Package Name: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0
    Application User Model ID: PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0!Python
    Target Path:
    C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0\python3.9.exe
msg406216 - (view) Author: (firewave) Date: 2021-11-12 14:07
The python3.9.exe file in the WindowsApps folder looks like a real executable and not a link. The script also backs that

python3 read_appexec.py "c:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2288.0_x64__qbz5n2kfra8p0\python3.9.exe"
Traceback (most recent call last):
  File "s:\Downloads\read_appexec.py", line 74, in <module>
    link = read_appexec_link(sys.argv[1])
  File "s:\Downloads\read_appexec.py", line 66, in read_appexec_link
    raise ctypes.WinError(ctypes.get_last_error())
OSError: [WinError 4390] The file or directory is not a reparse point.

Although I did this as Administrator so I can successfully list the directory I am not able to run that executable. It fails with "Access is denied.". Others are working fine.

idle3.9.exe - works
pip3.9.exe - works
python.exe - Access is denied.
python3.9.exe - Access is denied.
pythonw.exe - Access is denied.
pythonw3.9.exe - works

I opened both pythonw* files in the old "Dependency Walker" and it loaded them both fine and the dependencies appear to be identical by just taking a short look beside it not showing anything suspicious in them. The non-suffix version didn't run with "Failure starting the process. Access is denied (5).".
msg406217 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-12 15:05
> The python3.9.exe file in the WindowsApps folder looks like 
> a real executable and not a link.

As requested, and as shown in the output, you tested the file in Python 3.9's subdirectory of "WindowsApps", not in "WindowsApps" itself.

If the size in the `dir` output is 0 bytes, then it's an empty file without a reparse point. That would explain the stale link in the parent "WindowsApps" directory, since there's no source reparse point to clone. It doesn't explain why some of the links in the app's subdirectory weren't created correctly.
msg406221 - (view) Author: (firewave) Date: 2021-11-12 16:08
The "python3.9.exe" link is that one also points to the earlier version. All other links are fine.

Also all those files are 0 bytes - even the working ones.

Since I cannot even run the actual executable there seems to be an issue with the executables on top of the broken appexec link.

I also checked the actual executables and they are binary identical as well as having the identical permissions and timestamps.
msg406225 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-12 17:59
> Could it be possible that it might not be able to replace that link when it is currently in use?

That (or something like that) would be my suspicion. It's happened 
before, though as Eryk Sun says the link itself does not remain in use 
very long at all.

I think we've collected enough data about the broken link. You should be 
able to disable and re-enable the execution alias to get the correct one.
msg406241 - (view) Author: (firewave) Date: 2021-11-12 21:45
That didn't work. The link is wrong on the disk and that setting just controls the usage of it not the creation.
msg406257 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-12 23:38
You're right. You probably need to uninstall and reinstall for the 
subdirectories of WindowsApps.
msg406258 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-12 23:55
> Also all those files are 0 bytes - even the working ones.

The files should have no data, just appexec links. It's just an empty file if there's no link set.

> Since I cannot even run the actual executable there seems to be 
> an issue with the executables on top of the broken appexec link.

The executable under "%ProgramFiles%\WindowsApps" can only be executed unconditionally by processes that are running as SYSTEM, LOCAL SERVICE, or NETWORK SERVICE. Otherwise it requires an access token that contains a WIN://SYSAPPID security attribute that contains the package name. When executing an appexec link from the user's local "WindowsApps", CreateProcessW() uses the information in the link to construct this access token, and the call basically becomes CreateProcessAsUserW(hToken, ...). The EXEs and DLLs in the package can be executed from within the app itself. For example, subprocess.call('python') directly spawns the app's "python.exe". 

FYI, an access token is a kernel object that defines a security context, including the user, groups, privileges, capabilities, integrity level, trust level, security attributes, claim attributes, default owner, default permissions, associated logon session, and associated Windows session. Every process must have a primary token, which is permanently associated with the process. Each thread within a process may have an impersonation token, which is temporarily associated with the thread until it's replaced or removed.

> That didn't work. The link is wrong on the disk and that 
> setting just controls the usage of it not the creation.

Yes, the links in the app's subdirectory of the local "WindowsApps" are the basis for the ones that get created in the local WindowsApps when the alias is enabled. The system reads the source reparse point via FSCTL_GET_REPARSE_POINT and sets it on an empty file via FSCTL_SET_REPARSE_POINT. If the source appexec link is wrong or missing, then disabling and enabling the alias won't help. 

From the end user's perspective, the simplest solution is to uninstall the app; make sure its subdirectory is removed from the local "WindowsApps" (e.g. "%LocalAppData%\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0"); and install the app again.
msg406437 - (view) Author: (firewave) Date: 2021-11-16 21:59
Today 3.9.2544.0 was automatically installed on my system. Interestingly it did not update the broken "python3.9.exe" appexec links. The timestamp is still from the last update and the links still point to 3.9.2032.0. All other links were updated.
msg412902 - (view) Author: (firewave) Date: 2022-02-09 12:39
A few weeks ago I was able to re-install Python 3.9 and 3.10 after manually removing the %LocalAppData%\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.* and disabling/enabling the applinks manually and uninstalling/installing a few times to get rid of all left-over applinks on the disk.

I still had two entries by "App Installer" for python.exe and python3.exe in the UI. But I assume those have been there for ages now.

But as soon as the next update was installed both version broke again.

So I tried the same again and uninstalling and manually deleting the folders is no longer enough. I still have applinks in that folder which I cannot delete. And I can no longer install the apps at all. They fail with

Installation Failure: Windows failed to install the following update with error 0x80070020: 9P7QFQMJRFP7-PythonSoftwareFoundation.Python.3.9.

So now it's completely broken.
msg412910 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2022-02-09 14:42
It still unfortunately sounds like a Windows issue. All we do is specify that there should be a file there, we have nothing to do with creating or removing them. I'm not sure there's anything else we can do to help.

You could try reporting the issue using the Windows Feedback app. This at least will get it straight into the correct team's issue tracker, along with whatever logs you let it collect, though unless a number of people have been reporting it, it may not get a lot of attention unfortunately.
msg412916 - (view) Author: (firewave) Date: 2022-02-09 15:39
I figured it out.

I tried to delete the files using PowerShell and I got the following:

Remove-Item : Cannot remove item C:\Users\Username\AppData\Local\Microsoft\WindowsApps\python.exe: The
process cannot access the file 'C:\Users\Username\AppData\Local\Microsoft\WindowsApps\python.exe' because it
is being used by another process.

Using the "Resource Monitor" via the "CPU" tab and "Associated Handles" I was able the process which was using those. Somehow the handle search in "Process Explorer" was not returning this.

After closing the process I was able to remove the files and install Python again.

So it is exactly as I suspected before. When an update/uninstall is performed and the files are in use they cannot be updated/removed.

It should probably check this early and fail in case the files are in use so it doesn't perform a partial operation leaving the app unusable/not installable.
msg412917 - (view) Author: (firewave) Date: 2022-02-09 15:41
Actually "Process Explorer" finds the handle as well. No idea why it didn't come up in my last search.
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89947
2022-02-09 15:41:56firewavesetmessages: + msg412917
2022-02-09 15:39:24firewavesetmessages: + msg412916
2022-02-09 14:42:02steve.dowersetmessages: + msg412910
2022-02-09 12:39:39firewavesetmessages: + msg412902
2021-11-16 21:59:21firewavesetmessages: + msg406437
2021-11-12 23:55:02eryksunsetmessages: + msg406258
2021-11-12 23:38:13steve.dowersetmessages: + msg406257
2021-11-12 21:45:28firewavesetmessages: + msg406241
2021-11-12 17:59:37steve.dowersetmessages: + msg406225
2021-11-12 16:08:52firewavesetmessages: + msg406221
2021-11-12 15:05:30eryksunsetmessages: + msg406217
2021-11-12 14:07:24firewavesetmessages: + msg406216
2021-11-12 13:23:59eryksunsetfiles: + read_appexec.py

messages: + msg406211
2021-11-12 13:08:48eryksunsetfiles: - read_appexec.py
2021-11-12 12:15:15firewavesetmessages: + msg406206
2021-11-12 06:04:10eryksunsetfiles: + read_appexec.py
nosy: + eryksun
messages: + msg406190

2021-11-12 00:01:12steve.dowersetmessages: + msg406180
2021-11-11 21:41:33firewavesetmessages: + msg406178
2021-11-11 20:56:07steve.dowersetmessages: + msg406177
2021-11-11 20:17:23firewavecreate