classification
Title: Windows Store "stub" Python executables give confusing behaviour
Type: Stage: resolved
Components: Windows Versions: Python 3.9
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: brett.cannon, eryksun, paul.moore, steve.dower, tim.golden, uranusjr, zach.ware
Priority: normal Keywords:

Created on 2020-07-17 18:42 by paul.moore, last changed 2020-07-21 17:23 by eryksun. This issue is now closed.

Messages (12)
msg373845 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-07-17 18:42
First of all, I do know that this is an issue with the Windows Store distribution, rather than the python.org one. But (a) I don't know where to report a bug against the Store implementation except here, and (b) it's arguably a case of the Windows implementation "stealing" the Python command, so worth flagging against core Python.

The problem is that Windows 10 installs `python` and `python3` executables by default, which open the Windows Store offering to install Python. That's not a bad thing - it's nice to have Python available with the OS! Although (see below) hijacking the `python` and `python3` commands for this purpose has some problems.

But those stubs silently do nothing when run with command line arguments, and because the python.org distribution doesn't put Python on PATH by default, users end up with the stubs available even if they install python.org Python.

We're getting a lot of bug reports from users as a result of this, when they follow standard instructions like "execute `python -m pip`". With the stub on the path, this silently does nothing, even though the user has installed Python. This is a very bad user experience, and not one that projects like pip can do anything to alleviate, other than having extremely convoluted "how to run pip" commands:

"""
To run pip, type `python -m pip` if you're on Unix, and `py -m pip` on Windows. Unless you're using the Windows Store version when you should do `python -m pip` on Windows too. Or on Unix when you might need to do `python3 -m pip`. Or...
"""

I don't have a good answer to this issue. Maybe the Windows Store stubs could detect core Python and do something better if it's installed? Maybe the stubs should print an explanatory message if run with command line arguments? Maybe Store Python should be available on Windows via some less error-prone mechanism?

I'm pretty sure if a Linux distribution shipped a `python` command that didn't run Python the way users expected, there would be complaints. After all, we have PEP 394 that explicitly states how we expect Unix systems to work. Maybe we need something similar for Windows?
msg373846 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-07-17 18:44
See, for example, https://github.com/pypa/packaging-problems/issues/379 as an illustration of the user (and project maintainer!) confusion this causes.
msg373847 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2020-07-17 18:50
Closing as "third-party" as those stubs are controlled by Microsoft Windows and has nothing to do with us as a project beyond that they install the Windows Store copy that Steve uploads (although this is all feedback to Steve who has connections on the Windows team).

And I will say I believe there are shims on some Linux distros like Ubuntu for things like venv, pip, etc. which are not included in-box which people typically expect to be there.
msg373849 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-07-17 19:01
I thought that might be the answer. But does anyone know where I can repost this as an issue against the Store distribution? I'm happy to report there if I can find out how...

Hopefully Steve can point me in the right direction.
msg373851 - (view) Author: Tzu-ping Chung (uranusjr) * Date: 2020-07-17 19:33
FWIW, I tweeted about this a while ago (January) and IIRC Steve said there’s plan to change that. https://twitter.com/uranusjr/status/1212450480917340160
msg374026 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-20 20:58
It already returns a non-zero exit code (should be (IIRC) 9009 to match the built-in cmd.exe result), and I've been trying to get the message added for at least a year now.

Unfortunately, I can only push it so far before it has to work through the Windows team's process, and then there's nobody to push it along. All that could really be done from outside is to organise a "why don't you fix the dev experience" spam campaign, but I'm not going to condone that ;)
msg374028 - (view) Author: Tzu-ping Chung (uranusjr) * Date: 2020-07-20 21:26
What would be the best channel to raise this issue to the Windows team from the outside? It does not need to be a spam campaign, but it’d be nice if we could direct the affected users somewhere instead of pypa/packaging-problems and various issue trackers, where the Windows team wouldn’t see.
msg374030 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-20 22:16
It's already gone through the correct channels, so any other submissions will be duped by the triagers.

The best person to post at is me, but I've suffered enough for it :) I'm pushing.
msg374053 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-07-21 07:01
Understood that these would be duplicates, and I certainly don't want to put more pressure on you, but I think there would still be value in having a known reporting channel for people to say "I am also affected by this":

* MS Developers get a more objective feel for the scale of the issue.
* Users who report the issue can track progress (they will presumably get told *what* their report is a duplicate of, and can follow the underlying issue).
* Python package developers have something definite that they can offer to users.

My suspicion here is that MS simply doesn't have a user-visible support channel for the Store Python stubs, which is why we've ended up discussing this here.
msg374056 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-07-21 09:05
> there would still be value in having a known reporting channel

Windows 10 has a "Feedback Hub" to report problems and search for existing feedback that's similar. You could report a problem with the "AppInstaller" app. In this case, the app execution alias targets "AppInstallerPythonRedirector.exe". This is a GUI program, so CMD won't wait on it and set %errorlevel% if run from an interactive prompt -- not unless one uses `start /w` to force it to wait. (Sidebar: CMD determines that it's a GUI program by querying the PEB address and reading the process PEB via ReadProcessMemory... hack-o-rama.) 

IMO, the installer-redirector in this case should be a console application that prints a message to stdout to inform the user that it's opening the store to install Python. If it fails, it should print an error message to stderr and return a non-zero exit status.
msg374071 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-07-21 15:58
> Windows 10 has a "Feedback Hub" to report problems and search for existing feedback that's similar.

This is the correct channel, but ultimately the "real" issue tracker is private, so all you'll ever get is "not finished yet" that suddenly becomes "finished" (and generally that's updated on release, not on commit, because we've found that users get confused when something is claimed to be fixed but they're still seeing the issue).

> IMO, the installer-redirector in this case should be a console application that prints a message to stdout to inform the user that it's opening the store to install Python.

This is exactly what it does. The fact that it is SUBSYSTEM:WINDOWS instead of CONSOLE is what I'm trying to get fixed. It's not complicated, it's just not something that anyone can force through besides the team that owns the code, and they own a lot of other things that need fixing as well (and the release schedule doesn't allow for "just one little fix"). Shipping an OS is *hard* ;)
msg374075 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2020-07-21 17:23
> This is exactly what it does. The fact that it is SUBSYSTEM:WINDOWS

Sorry, I neglected to check:
 
    C:\>python3 script.py 2>&1 | more
    Python was not found but can be installed from the Microsoft Store: https://go.microsoft.com/fwlink?linkID=2082640
History
Date User Action Args
2020-07-21 17:23:05eryksunsetmessages: + msg374075
2020-07-21 15:58:15steve.dowersetmessages: + msg374071
2020-07-21 09:05:58eryksunsetnosy: + eryksun
messages: + msg374056
2020-07-21 07:01:17paul.mooresetmessages: + msg374053
2020-07-20 22:16:05steve.dowersetmessages: + msg374030
2020-07-20 21:26:38uranusjrsetmessages: + msg374028
2020-07-20 20:58:26steve.dowersetmessages: + msg374026
2020-07-17 19:33:27uranusjrsetnosy: + uranusjr
messages: + msg373851
2020-07-17 19:01:33paul.mooresetmessages: + msg373849
2020-07-17 18:50:03brett.cannonsetstatus: open -> closed

nosy: + brett.cannon
messages: + msg373847

resolution: third party
stage: resolved
2020-07-17 18:44:13paul.mooresetmessages: + msg373846
2020-07-17 18:42:52paul.moorecreate