Issue34977
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-10-13 22:43 by steve.dower, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Pull Requests | |||
---|---|---|---|
URL | Status | Linked | Edit |
PR 10245 | merged | steve.dower, 2018-10-30 15:55 | |
PR 10677 | merged | steve.dower, 2018-11-23 14:56 | |
PR 11019 | merged | vstinner, 2018-12-07 11:32 | |
PR 11021 | merged | vstinner, 2018-12-07 11:51 | |
PR 11027 | merged | steve.dower, 2018-12-07 15:06 | |
PR 11028 | merged | steve.dower, 2018-12-07 15:08 | |
PR 11029 | merged | steve.dower, 2018-12-07 16:03 | |
PR 11030 | merged | steve.dower, 2018-12-07 16:25 | |
PR 11092 | merged | steve.dower, 2018-12-11 04:37 | |
PR 11093 | merged | miss-islington, 2018-12-11 04:56 | |
PR 5812 | izbyshev, 2019-10-18 00:21 |
Messages (42) | |||
---|---|---|---|
msg327679 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-10-13 22:43 | |
The Windows Store can now install unrestricted apps, which means we can provide the Python interpreter in there. Advantages: * far more reliable installation * can have specific executables on PATH * automatic updates * other apps can act as extensions (with permission), so things like Mu could also become store apps in the future Disadvantages: * only --user installs for pip will work * harder to find/modify installed files I think for a (very significant) subset of our users, this will be a much better experience than downloading our current installer. It also has the advantage of making "Python" appear in searches in the start menu, which will link directly to a one-click "Install" button. The biggest issue is likely to be pip not installing with --user by default, as well as subsequent issues with apps that require users to navigate to their %AppData% directories manually (since these will be redirected to different locations). But until we get a package available for testing, it will be hard to figure out what needs fixing. Given this is just distribution and not a new platform, I plan to enable it for Python 3.7 (probably 3.7.2, but it may be possible to bundle 3.7.1 as-is for testing). |
|||
msg327691 - (view) | Author: Karthikeyan Singaravelan (xtreak) * ![]() |
Date: 2018-10-14 06:02 | |
+1 for reliable installation. There were some cases (issue33220, issue30944) where the installer was detected as a Virus by some vendors which would be reduced since this is installed from the App store as a verified app. I hope this will also reduce the scenarios where users need to resort to bug tracker with log files when there are installation failures [1] providing a more smooth one click install. There is a long standing open issue [2] to make `pip install --user` as a default which I hope gets resolved and there is an issue with using `pip install --user` inside a virtualenv which seems to block the earlier issue [3] Thanks a lot for this! [1] https://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=windows+msi+install&submit=search&status=-1%2C1%2C2%2C3 [2] https://github.com/pypa/pip/issues/1668 [3] https://github.com/pypa/pip/issues/5702 |
|||
msg328146 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-10-20 15:19 | |
I'm 99% sure that with issue34725 I'll be able to make "pip --user" the default when using this install, as well as making the commands globally available (i.e. "python[3[.8]]", "pip[3[.8]]", "idle[3[.8]]" will work everywhere). |
|||
msg328147 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-10-20 15:19 | |
(And I'll deal with that 1% at the PyPA sprints next weekend. We are due for a new pip insertion anyway, so I guess that'll come for 3.7.2.) |
|||
msg330166 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-11-21 00:29 | |
For people watching the bug, this is ready for review. Most of the change is adding a "layout" script (roughly the equivalent of "make install" for Windows, but with more flexibility to generate different structures, precompile stuff, etc.) About the only CPython change is to venv. I added a new launcher variation that looks for pyvenv.cfg and runs the python.exe listed in that, so now we don't have to copy any DLLs into the venv at all (it's needed because the old one would try to LoadLibrary DLLs inside the store package, which is not allowed, but it also fixes my #1 complaint about venv which is that upgrading the base Python breaks all venvs). There are also a few tweaks to the startup process, including extending the existing __PYVENV_LAUNCHER__ env variable from macOS to Windows as well, though it's handled in a different location. And a couple of minor tweaks to tests - issues discovered now that it's pretty easy to generate a "real" layout to run tests in. I can break this up into a couple of PRs if that's preferred, though it doesn't really gain anything. If you're not interested in the new layout script, stop reading the PR after you finish launcher.c :) |
|||
msg330359 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-11-23 19:57 | |
If anyone would like to try this out early, here's some instructions for a build I just made. Enable sideloading apps on your machine (see https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development - may already be done, sideload or developer mode should do) In a regular (non-admin) PowerShell prompt in a temporary directory, run these commands: iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34540/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip Expand-Archive .\python37.zip . Add-AppxPackage appx\python.msix (The first may take a while as it downloads approx. 80MB. I'm intending to leave the test suite out of the final package, which should bring it down closer to 40MB.) Now play with it: $> python.exe $> python3.exe $> python3.7.exe $> pip.exe $> pip3.exe $> pip3.7.exe $> idle.exe $> idle3.exe $> idle3.7.exe (Or press Win+S and search for new entries) To remove it: Remove-AppxPackage (Get-AppxPackage *Python.3.7) (delete the downloaded files) Right now, the py.exe launcher can't find it, nor can any tools that rely on enumerating the registry - directly requesting the reg keys will return the right values though. VS Code seems to pick it up just fine though, I guess because it's on PATH. |
|||
msg330452 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-11-26 18:41 | |
For my own reference, some issues I found installing this package on a clean machine: * requires Developer Mode (until I get the sccd file signed - just sideloading is not sufficient) * requires vcruntime140.dll to be included in the package (workaround is to install the redistributable from https://visualstudio.microsoft.com/downloads/#title-39324) * venv needs to treat .pdb files as binary |
|||
msg330643 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-11-29 00:24 | |
Updated testing instructions - this one should work with just sideloading permissions I think. iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34760/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip Expand-Archive .\python37.zip . Add-AppxPackage appx\python.msix To remove: Remove-AppxPackage (Get-AppxPackage *Python.3.7) (delete the downloaded files) |
|||
msg331265 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 05:09 | |
New changeset 468a15aaf9206448a744fc5eab3fc21f51966aad by Steve Dower in branch 'master': bpo-34977: Add Windows App Store package (GH-10245) https://github.com/python/cpython/commit/468a15aaf9206448a744fc5eab3fc21f51966aad |
|||
msg331266 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 05:09 | |
New changeset 253209149389e6793a052034e1f2d97691086f18 by Steve Dower in branch '3.7': [3.7] bpo-34977: Add Windows App Store package (GH-10245) https://github.com/python/cpython/commit/253209149389e6793a052034e1f2d97691086f18 |
|||
msg331267 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 05:15 | |
Making the release experimental as part of the next 3.7 update was approved by Ned (over email), so I merged the build. As soon as we snap for the RC I'll kick off an update and make the store page public, and hopefully can promote it enough to get eyes on it. Unfortunately, I discovered as part of a test submission that the minimum Windows version matters, and it's a version that hasn't been rolled out fully yet because of some bugs, so there may not be that many people who can use it this first time. But that will improve over time, and I'm sure I can find enough people to flush out issues before the next release (anyone on the Windows Insiders program should be fine). |
|||
msg331300 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 11:46 | |
> Making the release experimental as part of the next 3.7 update was approved by Ned (over email), so I merged the build. I suggest to revert this change from Python 3.7: it broke all Windows buildbots, see bpo-35437. It doesn't seem like a good idea to push such change between a release candidate and a final version :-( |
|||
msg331301 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 11:57 | |
New changeset cb0b78a070ea3b704416e74f64046178ae0dff3e by Victor Stinner in branch 'master': Revert "bpo-34977: Add Windows App Store package (GH-10245)" (GH-11019) https://github.com/python/cpython/commit/cb0b78a070ea3b704416e74f64046178ae0dff3e |
|||
msg331304 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 12:49 | |
> I suggest to revert this change from Python 3.7: it broke all Windows buildbots, see bpo-35437. I reverted the change in master and I will merge PR 11021 once tests pass to revert in 3.7 as well. > It doesn't seem like a good idea to push such change between a release candidate and a final version :-( I started a discussion on python-dev: https://mail.python.org/pipermail/python-dev/2018-December/155932.html |
|||
msg331310 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 13:31 | |
New changeset 783b794a5e6ea3bbbaba45a18b9e03ac322b3bd4 by Victor Stinner in branch '3.7': Revert "[3.7] bpo-34977: Add Windows App Store package (GH-10245)" (GH-11021) https://github.com/python/cpython/commit/783b794a5e6ea3bbbaba45a18b9e03ac322b3bd4 |
|||
msg331311 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 13:32 | |
> Revert "[3.7] bpo-34977: Add Windows App Store package (GH-10245)" (GH-11021) That should repair buildbots, so I remove "release blocker" priority. |
|||
msg331316 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 14:13 | |
We haven't done RC yet - why is pushing this now so bad? Most of the change is purely additive and does not modify the code base, the rest is Windows-only or tests. I'll add checks for outdated build machines, but there has to be a release of this or it remains untestable, so it'll either be from my branch or from the release branch. |
|||
msg331321 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 14:50 | |
> We haven't done RC yet - why is pushing this now so bad? I reverted your change because it broke all Windows buildbots, and we have a policy to revert a change if buildbots turn red and no fix can be found in 2 hours: https://pythondev.readthedocs.io/ci.html#revert-on-fail > Most of the change is purely additive and does not modify the code base, the rest is Windows-only or tests. nitpicking: Your change modify venv and two tests, it's not purely Windows-only :-) By the way, as I wrote in my email, I would prefer to see the venv+tests change pushed in separated change (and explain why you do these changes). > I'll add checks for outdated build machines, but there has to be a release of this or it remains untestable, so it'll either be from my branch or from the release branch. Sorry, I don't know Windows App Store, I don't understand the restriction "there has to be a release of this or it remains untestable". Would you mind to elaborate? Do you mean that buildbots must upgrade to VS 2017? Can Python 3.6 and 3.7 be compiled without any issue on VS 2017? Maybe send an email to buildbot-status if you want to see more buildbots running VS 2017 rather than VS 2015. -- There was a small bug in your PR, in PC/launcher.c, fixed by commit 4c49da0cb7434c676d70b9ccf38aca82ac0d64a9 (bpo-35436): + if (executable == NULL) { + error(RC_NO_MEMORY, L"A memory allocation failed"); + } |
|||
msg331322 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-07 14:51 | |
Since the first attempt broke all Windows buildbot workers, it would be nice to use a custom build before merging a new attempt :-) See instructions in the devguide: https://devguide.python.org/buildbots/?highlight=custom#custom-builders Note: Pablo used buildbot-custom to test a revert of the App Store change. |
|||
msg331328 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 15:50 | |
>> We haven't done RC yet - why is pushing this now so bad? > I reverted your change because it broke all Windows buildbots, and we have a policy to revert a change if buildbots turn red and no fix can be found in 2 hours I don't dispute that policy, but you said it shouldn't go in between RC and a release, which it isn't. > nitpicking: Your change modify venv and two tests, it's not purely Windows-only :-) More nitpicking: the venv changes only affect Windows. Which makes it purely Windows and tests, exactly as I said. Adding smiles doesn't save you from having to read :) > Sorry, I don't know Windows App Store, I don't understand the restriction "there has to be a release of this or it remains untestable". Would you mind to elaborate? Sure. You can start by reading this thread, but the summary is that App Store apps (regardless of platform) generally have to go through an approval process before they can be installed. Without going through this process, they can't be tested fully in context. I believe we should make a true release and put that through the process, rather than me just doing one out of my branch. I provided instructions for developer testing above, which covers about 80% of what needs to be checked. The rest can only be tested by real use of a proper release. |
|||
msg331350 - (view) | Author: Terry J. Reedy (terry.reedy) * ![]() |
Date: 2018-12-07 20:29 | |
Various questions and comments: 0. Steve: For more than one reason, I want this to be and remain an added alternative rather than a replacement of the current installer directly available on python.org. Can you assure us that the latter are not going away? I would have the same concern about adding something to the Apple/Mac store. 1. Ned: Since enhancements are not normally backported, it would be nice to have your approval, in whatever form it took, recorded here and not just in private email. I give a reason below. 2. Installers are always experimental until tested by multiple users on multiple machines. Even then, installation specific issues can arise. As I remember, the current installer, when new in 3.5, needed a shakedown period. One installer-specific issue was the initial working directory. To have a Windows Store Python ready for 3.8, we need to start now. There was once an issue where one of the installers failed to include one idlelib file. I believe it was help.html. I since added idle_test/test_help.py, which would now fail with something like "No such file or directory: 'C:\\Programs\\Python37\\Lib\\idlelib\\help.html'".) Repository testing does not test the actual binaries that users install, in their production context. Steve, assuming that I can access the Store, should I be able to install both PSF 3.7.2 and Windows Store 3.7.2 simultaneously? So I could test both IDLEs that beginners might install? 3. Anyone: Though it is not essential that I know, I am curious what 'restricted' versus 'non restricted' mean to Windows Store, and why Python can now go there. My search only came up with age restrictions, which python does not need (in my opinion). 4. While most PRs should be complete in themselves, reasons such as ease of review may suggest splitting one conceptual Pr into pieces. They just have to be reviewed together, or with expectation of a needed followup (and a revert if it does not appear). I have done this to get readable diffs for major refactorings that moved and edited multiple chunks of code in a large file. |
|||
msg331352 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 20:43 | |
Thanks Terry, great questions. > I want this to be and remain an added alternative rather than a replacement of the current installer directly available on python.org. Can you assure us that the latter are not going away? Absolutely. The current installer is not going anywhere, and there's no point considering the possibility until we've completely dropped support for Windows 10 1803 and earlier (which was only released this year, so dropping support is a long way out yet). > Installers are always experimental until tested by multiple users on multiple machines. Even then, installation specific issues can arise. ... To have a Windows Store Python ready for 3.8, we need to start now. Thanks for the vote of support for this point. > Steve, assuming that I can access the Store, should I be able to install both PSF 3.7.2 and Windows Store 3.7.2 simultaneously? So I could test both IDLEs that beginners might install? Yes, absolutely. (To be clear, both are PSF approved and under the PSF name/copyright/etc., so I've been calling them "traditional" and "Store" installers when I talk about them.) Also, I've been testing IDLE regularly, since one of the advantages of being in the Store is that we can safely put idle[3[.7]].exe on PATH. It definitely works :) I've also set up tests that run from an "installed" layout, rather than the source tree, and those are all passing as well (with one of the trivial test fixes in the PR). > I am curious what 'restricted' versus 'non restricted' mean to Windows Store, and why Python can now go there. In general, all app stores disallow their apps from doing things that may break or corrupt the machines they are installed on. Apps are self-contained and can only operate outside of their boundaries through carefully designed permissions/capabilities (which often prompt the user before continuing). Recently, Microsoft added support for "full trust" apps, which can bypass many of these restrictions. The aim is to allow traditional applications to swap their MSIs for being installed through the store, which can then ensure trustworthiness of at least the installers. (For example, a Store app is *never* going to install something into System32 or modify operating system files, whether it's full trust or not.) Full trust allows us to do things like launching non-app store processes and creating raw sockets through legacy APIs, so we can very easily move Python into a full trust package rather than having to rewrite it using approved APIs (a project that's been attempted a few times, but always loses low-level functionality that is considered a core part of Python - hence my long-term desire to make the low-level parts optional and have higher-level abstractions be the guaranteed parts). > While most PRs should be complete in themselves, reasons such as ease of review may suggest splitting one conceptual Pr into pieces. They just have to be reviewed together, or with expectation of a needed followup (and a revert if it does not appear). I have done this to get readable diffs for major refactorings that moved and edited multiple chunks of code in a large file. I agree that it's valuable and I've done it internally plenty of times. Unfortunately, no matter how much people repeatedly say that a PR for CPython is part of a chain, there always seems to be pushback from people who ignore that and won't approve a critical prerequisite because there's "no need". There's also the point that many of the changes were not known to be needed before the others were made. The pre-squash history is very interleaved between the various changes, and extracting them into separate PRs that totally stand alone is nearly as much work as solving the issues in the first place. I made the call this time to keep them all together and direct reviewers' attention towards the relevant parts. It turns out that didn't attract reviewers, so I'll come up with a new strategy next time I want to get people's attention on a change (maybe Buzzfeed style headlines? :) ) |
|||
msg331356 - (view) | Author: Paul Moore (paul.moore) * ![]() |
Date: 2018-12-07 21:18 | |
> > Steve, assuming that I can access the Store, should I be able to install both PSF 3.7.2 and Windows Store 3.7.2 simultaneously? So I could test both IDLEs that beginners might install? > > Yes, absolutely. (To be clear, both are PSF approved and under the PSF name/copyright/etc., so I've been calling them "traditional" and "Store" installers when I talk about them.) I'd like to expand on this question a little. The first thing I'll note is that I'm not aware of any documentation on the Windows Store installer, and how it works. It's possible there's a docs patch in here somewhere, but to be honest, there are 8 PRs linked to this issue and I don't have the time to check them all right now. How will the Windows Store install work alongside the PSF one? Will it work the same, in that neither will go on PATH unless the user explicitly requests it? If I have PSF Python 3.7.1 and Store Python 3.7.2 simultaneously installed, which will py.exe -3.7 select? Can I influence it to choose the other one? Same question for 3rd party tools like "virtualenv -p 3.7" - I know it's not up to us to deal with virtualenv issues, but conversely I do think we have a responsibility to not change assumptions that 3rd party tools rely on. In terms of PEP 514, will the Store installer use a different tag, or some other way of having registry entries independent of the PSF installer? Basically, I'm not clear on how having the Store install and the PSF install simultaneously present will work - very few tools in my experience really support PEP 514 (not even the `py` launcher) - and frankly, if we're going to break tool assumptions that there's only one official "3.7" present at any time, we're on very shaky ground claiming that PEP 514 warned them if we haven't even updated our own supplied tool! (I know that there's lots of ways people can have multiple versions right now - PSF plus Anaconda, for example - but it's fine to say "you're mixing distributions, you're on your own" in that case. If the PSF is publishing two versions of 3.7, which can be simultaneously installed, we can't really use that argument. Paul |
|||
msg331357 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 21:40 | |
> The first thing I'll note is that I'm not aware of any documentation > on the Windows Store installer, and how it works. Right now, correct. > How will the Windows Store install work alongside the PSF one? It will go into PATH under the same names as on other platforms (python, python3, python3.7, and same for pip and idle). The Windows Store makes it possible to do this reliably, so it's just on by default (and configurable through OS-provided UI for selecting the app to respond to a certain name). If you have already installed it traditionally and enabled the PATH modification, that will take precedence. So you'll get the MSI installed Python. If you've modified PATH by hand, it depends on where you put it. > If I have PSF Python 3.7.1 and Store Python > 3.7.2 simultaneously installed, which will py.exe -3.7 select? Can I > influence it to choose the other one? Same question for 3rd party > tools like "virtualenv -p 3.7" Right now, py.exe will not detect it. This is an unavoidable limitation right now (specifically, enumeration of classic app compatibility keys does not work, though direct access does), but it may become avoidable in the future (and being able to say "Python needs it" is actually going to be useful for removing the limitation). As usual, our recommended way to influence choice of Python is to specify it. "virtualenv -p 3.7" will find a traditionally installed Python first, but if it's not there and the store one is it *might* find the store one (it depends whether it requests the specific registry key or tries to enumerate all of them - again, I've got people looking into enabling enumeration, but we're talking about adding another feature to Windows specifically for Python, and those take time). > Basically, I'm not clear on how having the Store install and the PSF > install simultaneously present will work - very few tools in my > experience really support PEP 514 (not even the `py` launcher) - and > frankly, if we're going to break tool assumptions that there's only > one official "3.7" present at any time, we're on very shaky ground > claiming that PEP 514 warned them if we haven't even updated our own > supplied tool! (I know that there's lots of ways people can have > multiple versions right now - PSF plus Anaconda, for example - but > it's fine to say "you're mixing distributions, you're on your own" in > that case. If the PSF is publishing two versions of 3.7, which can be > simultaneously installed, we can't really use that argument. In all the cases I've tested, regular Python install shows up first. If it's not installed, the Store app shows up much of the time. This feels like the right design (with the caveat about "much of the time" requiring changes outside of our direct control), but without actually giving it to people in all their messy configurations there is literally no way to be sure. A regular install does not create "python3" and "python3.7" shortcuts on PATH, so if you use those you'll prefer the store app. But we've never used those, so there shouldn't be any change to existing behaviour. And I've never really been opposed to adding them to the Python installer, I just haven't done the work (and neither has anyone else) - I'd still be opposed to modifying PATH from the traditional installer though, as all of those problems remain. |
|||
msg331358 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 21:49 | |
> How will the Windows Store install work alongside the PSF one? Oh, and since I didn't mention it in my direct answer, the Store install goes to a totally different location (per machine, no elevation required), as do packages installed using pip (automatically per user). Nothing is shared between the two installs at all. |
|||
msg331363 - (view) | Author: Paul Moore (paul.moore) * ![]() |
Date: 2018-12-07 23:12 | |
Thanks for the clarification. > In all the cases I've tested, regular Python install shows up first. If it's not installed, the Store app shows up much of the time. This feels like the right design (with the caveat about "much of the time" requiring changes outside of our direct control), but without actually giving it to people in all their messy configurations there is literally no way to be sure. Agreed, it does sound like it's probably the correct approach. But I'd strongly recommend that as much of the behaviour as possible gets documented as part of this change. There are a *lot* of projects out there that write code for Windows based on a relatively superficial understanding of all of the complexities that come into play here - most registry scanning code is based on guesswork and reverse engineering of how things worked before PEP 514 (I should know, I wrote rather a lot of it!!!) and plays fast and loose with even things like 32-bit/64-bit registry overlaying (it's late and I'm tired, sorry for not using the accurate term). Settig projects like virtualenv, tox, nox, pew, pipenv etc up for new bug reports associated with store installs, without any sort of guidance as to what's going on in the official docs, seems unfriendly at best. (I'm thinking of a section in "Python Setup and Usage" covering the store installer - that in addition to the changes in https://docs.python.org/3.7/using/windows.html#installation-steps needed to mention that there's now five installers available). > A regular install does not create "python3" and "python3.7" shortcuts on PATH, so if you use those you'll prefer the store app. But we've never used those, so there shouldn't be any change to existing behaviour. And I've never really been opposed to adding them to the Python installer, I just haven't done the work (and neither has anyone else) - I'd still be opposed to modifying PATH from the traditional installer though, as all of those problems remain. The "pythonup" project (https://github.com/uranusjr/pythonup-windows) creates versioned copies of python - python3.7.exe. Again, it's a non-standard project, so not technically our problem, but I'd be interested in how it would interact with the Store python. I do appreciate that we need to get the store installer out there in order to test things like this, and that initially at least there aren't going to be a huge number of people using the store installer over the traditional one (and likely vanishingly small numbers of people using both in parallel). But conversely, there's a class of problems that will be reported to other projects rather than to us, and I think we should be trying to support those other projects in dealing with this change, not simply leaving it up to them to find out about it (I doubt they'd even know to ask "did you use the store install or the traditional one" when diagnosing a problem). In general, I don't think we've done that great a job of publicising the various install options on Windows - we still get pip users trying to install packages into the embedded distribution, which they are using as if it were a standalone interpreter, and as far as I can tell essentially no-one knows about the nuget packages for Python. Let's not repeat the mistakes we've made previously with this new distribution option. |
|||
msg331368 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-07 23:45 | |
> In general, I don't think we've done that great a job of publicising > the various install options on Windows - we still get pip users trying > to install packages into the embedded distribution, which they are > using as if it were a standalone interpreter, and as far as I can tell > essentially no-one knows about the nuget packages for Python. Let's > not repeat the mistakes we've made previously with this new > distribution option. Yeah, and these are all essentially my fault (though I documented the embeddable package relatively well, modulo a few uses I hadn't imagined). Maybe my holiday writing project can be some better docs for choosing how to install Python, especially since if/when this merges we'll finally have a good answer for "installing from a build without the installer" (using the PC/layout script, which *sigh* also needs a big fat manual). I feel like right now there's enough here that isn't quite locked down and I don't want to guarantee it. For example, in a future release of Windows we may be able to enable it to write to the actual %APPDATA% folder instead of an app-private one - should we document the current paths and then "break" it in the future? Or should we document that using the "Reset app" features will clean up everything fully now, knowing that if we change the %APPDATA% location later then _that_ will break? So for now, I want an unguaranteed store package that can be used but may change over time (and I'll update the text on the page to reflect that). Now is not the time to lock in all aspects of how it works, exactly as you say. But please hold me accountable for adding specific docs for the first 3.8 release - hopefully it's clear by then (if not earlier) what behavior we're happy with. |
|||
msg331377 - (view) | Author: Paul Moore (paul.moore) * ![]() |
Date: 2018-12-08 11:06 | |
> Yeah, and these are all essentially my fault (though I documented the embeddable package relatively well, modulo a few uses I hadn't imagined). Maybe my holiday writing project can be some better docs for choosing how to install Python, especially since if/when this merges we'll finally have a good answer for "installing from a build without the installer" (using the PC/layout script, which *sigh* also needs a big fat manual). Sorry about that - I wasn't trying to make you feel guilty! And yes, the embeddable package docs are very good. The problem with that one is mostly that people don't *read* the docs (but having them to point to makes responding to pip issues much easier, so you have my gratitude there :-) > I feel like right now there's enough here that isn't quite locked down and I don't want to guarantee it. For example, in a future release of Windows we may be able to enable it to write to the actual %APPDATA% folder instead of an app-private one - should we document the current paths and then "break" it in the future? Or should we document that using the "Reset app" features will clean up everything fully now, knowing that if we change the %APPDATA% location later then _that_ will break? > > So for now, I want an unguaranteed store package that can be used but may change over time (and I'll update the text on the page to reflect that). Now is not the time to lock in all aspects of how it works, exactly as you say. But please hold me accountable for adding specific docs for the first 3.8 release - hopefully it's clear by then (if not earlier) what behavior we're happy with. In which case, let's have a placeholder note in the docs, saying that the Windows Store installer exists, but it's experimental and may have unexpected interactions with other tools that expect a standard Python install. Users trying the Windows Store installer should be prepared for such issues, and we'd appreciate reports so that we can address them. Add that to the release announcement as well for extra visibility. |
|||
msg331387 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-08 15:46 | |
> In which case, let's have a placeholder note in the docs, saying that > the Windows Store installer exists, but it's experimental and may have > unexpected interactions with other tools that expect a standard Python > install. Users trying the Windows Store installer should be prepared > for such issues, and we'd appreciate reports so that we can address > them. Add that to the release announcement as well for extra > visibility. Sure, but first I need PR 11029 (and 11030 the backport) approved, so that I can rebase and work on redoing the other half of the change. Victor seems to have disappeared for the weekend, but I made all his requested changes. |
|||
msg331517 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-10 15:55 | |
I'm just waiting on Victor to sign off on PR 11029 (or someone else to do it) so I can merge it and rebase PR 11027. Hopefully then someone will review it today so that we don't have to hold up 3.7.2rc1 any longer. |
|||
msg331518 - (view) | Author: STINNER Victor (vstinner) * ![]() |
Date: 2018-12-10 15:57 | |
Sorry, I don't have the bandwidth to provide a proper review. |
|||
msg331519 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-10 16:11 | |
Okay, I'll merge the first part now so I can rebase the main one and *hopefully* someone will be willing to review it now it's smaller. Otherwise, I think we're best to go with buildbots and all the testing I've already done. This is my sole job today, so I'll keep an eye on the buildbots. I'll also get the docs updated as Paul suggested for the main PRs. |
|||
msg331520 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-10 16:11 | |
New changeset 1c3de541e64f75046b20cdd27bada1557e550bcd by Steve Dower in branch 'master': bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029) https://github.com/python/cpython/commit/1c3de541e64f75046b20cdd27bada1557e550bcd |
|||
msg331521 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-10 16:11 | |
New changeset b264c609853eae9dbb45c6dbee11e84ae3927e88 by Steve Dower in branch '3.7': [3.7] bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029) https://github.com/python/cpython/commit/b264c609853eae9dbb45c6dbee11e84ae3927e88 |
|||
msg331530 - (view) | Author: Jeremy Kloth (jkloth) * | Date: 2018-12-10 18:27 | |
See also bpo-35450: venv module doesn't create a copy of python binary by default |
|||
msg331534 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-10 20:36 | |
Paul (and anyone else) - the below link should go directly to just the commit with the docs update. I did a slight rearrangement of the install docs to make the options clearer, and wrote up *just* enough info on nuget to help people use it right (I hope). Similarly for the Store package. Any comments appreciated. Still aiming to get this in for 3.7.2rc1 today. https://github.com/python/cpython/pull/11027/commits/7d7ddbc41bd30690de8c6227cebc41c4e0f3cf88 I've also kicked off a custom buildbot run against the 3.7 version of the change. The version in master ran earlier and was clean. Plus I'm running test builds on the release machine, so it should be ready to go. |
|||
msg331553 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-11 00:25 | |
Thanks, Paul. Appreciate it :) Doing a final call before I hit merge in an hour or so. Right now: * full release build works * full test pass on all CI + custom buildbots work * prior test package made it to the store * docs clearly show it may change again Any last pushback? |
|||
msg331560 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-11 02:53 | |
New changeset 0cd6391fd890368ea1743dac50c366b42f2fd126 by Steve Dower in branch 'master': bpo-34977: Add Windows App Store package (GH-11027) https://github.com/python/cpython/commit/0cd6391fd890368ea1743dac50c366b42f2fd126 |
|||
msg331566 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-11 03:58 | |
New changeset 0e4ad88ff8956d9289ab0a1314636ac15b374459 by Steve Dower in branch '3.7': bpo-34977: Add Windows App Store package (GH-11027) https://github.com/python/cpython/commit/0e4ad88ff8956d9289ab0a1314636ac15b374459 |
|||
msg331571 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-11 04:40 | |
Luckily, just noticed that I introduced a bug as part of splitting up the original PR into two. It wasn't going to have an impact until the package was properly installed from the store, but without the proper directory resolution in there (since the variable I removed is never defined now), in some scenarios it wouldn't be possible to launch Python from a process outside of the app package, such as a virtual environment. Testing in a proper app container is difficult, but I'll see if I can come up with a way to do it. It might require a custom buildbot, so I'll see whether there's a way I can do that (historically we've not been able to wire internal work machines - including Azure hosted ones - into other CI systems, but I'll try again). |
|||
msg331572 - (view) | Author: Steve Dower (steve.dower) * ![]() |
Date: 2018-12-11 04:56 | |
New changeset d5a6a389d492c5e3d7933bafbd5252fd86ac4d49 by Steve Dower in branch 'master': bpo-34977: Remove unused preprocessor definition (GH-11092) https://github.com/python/cpython/commit/d5a6a389d492c5e3d7933bafbd5252fd86ac4d49 |
|||
msg331573 - (view) | Author: miss-islington (miss-islington) | Date: 2018-12-11 05:15 | |
New changeset 9bb306d586e3f1a48db40bd9519412de4fff3ee8 by Miss Islington (bot) in branch '3.7': bpo-34977: Remove unused preprocessor definition (GH-11092) https://github.com/python/cpython/commit/9bb306d586e3f1a48db40bd9519412de4fff3ee8 |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:07 | admin | set | github: 79158 |
2019-12-13 03:00:51 | cheryl.sabella | link | issue32907 superseder |
2019-10-18 00:21:36 | izbyshev | set | pull_requests: + pull_request16379 |
2018-12-12 23:45:02 | steve.dower | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
2018-12-11 05:15:01 | miss-islington | set | nosy:
+ miss-islington messages: + msg331573 |
2018-12-11 04:56:19 | miss-islington | set | pull_requests: + pull_request10325 |
2018-12-11 04:56:11 | steve.dower | set | messages: + msg331572 |
2018-12-11 04:40:53 | steve.dower | set | messages: + msg331571 |
2018-12-11 04:37:03 | steve.dower | set | pull_requests: + pull_request10324 |
2018-12-11 03:58:54 | steve.dower | set | messages: + msg331566 |
2018-12-11 02:53:09 | steve.dower | set | messages: + msg331560 |
2018-12-11 00:25:18 | steve.dower | set | messages: + msg331553 |
2018-12-10 20:36:09 | steve.dower | set | messages: + msg331534 |
2018-12-10 18:27:24 | jkloth | set | nosy:
+ jkloth messages: + msg331530 |
2018-12-10 16:11:37 | steve.dower | set | messages: + msg331521 |
2018-12-10 16:11:26 | steve.dower | set | messages: + msg331520 |
2018-12-10 16:11:09 | steve.dower | set | messages: + msg331519 |
2018-12-10 15:57:28 | vstinner | set | messages: + msg331518 |
2018-12-10 15:55:24 | steve.dower | set | messages: + msg331517 |
2018-12-08 19:15:31 | terry.reedy | set | nosy:
- terry.reedy |
2018-12-08 15:46:30 | steve.dower | set | messages: + msg331387 |
2018-12-08 11:06:52 | paul.moore | set | messages: + msg331377 |
2018-12-07 23:45:09 | steve.dower | set | messages: + msg331368 |
2018-12-07 23:12:39 | paul.moore | set | messages: + msg331363 |
2018-12-07 22:32:49 | ethan smith | set | nosy:
+ ethan smith |
2018-12-07 21:49:17 | steve.dower | set | messages: + msg331358 |
2018-12-07 21:40:09 | steve.dower | set | messages: + msg331357 |
2018-12-07 21:18:49 | paul.moore | set | messages: + msg331356 |
2018-12-07 20:43:45 | steve.dower | set | messages: + msg331352 |
2018-12-07 20:29:03 | terry.reedy | set | messages: + msg331350 |
2018-12-07 16:25:41 | steve.dower | set | pull_requests: + pull_request10266 |
2018-12-07 16:03:03 | steve.dower | set | pull_requests: + pull_request10265 |
2018-12-07 15:50:56 | steve.dower | set | messages: + msg331328 |
2018-12-07 15:08:15 | steve.dower | set | pull_requests: + pull_request10264 |
2018-12-07 15:06:35 | steve.dower | set | pull_requests: + pull_request10263 |
2018-12-07 14:51:59 | vstinner | set | messages: + msg331322 |
2018-12-07 14:50:13 | vstinner | set | messages: + msg331321 |
2018-12-07 14:13:53 | steve.dower | set | messages: + msg331316 |
2018-12-07 13:32:17 | vstinner | set | priority: release blocker -> messages: + msg331311 |
2018-12-07 13:31:45 | vstinner | set | messages: + msg331310 |
2018-12-07 12:49:01 | vstinner | set | messages: + msg331304 |
2018-12-07 11:57:46 | vstinner | set | messages: + msg331301 |
2018-12-07 11:51:16 | vstinner | set | pull_requests: + pull_request10258 |
2018-12-07 11:46:22 | vstinner | set | priority: normal -> release blocker nosy: + ned.deily, vstinner messages: + msg331300 |
2018-12-07 11:32:17 | vstinner | set | stage: commit review -> patch review pull_requests: + pull_request10255 |
2018-12-07 05:15:24 | steve.dower | set | stage: patch review -> commit review |
2018-12-07 05:15:16 | steve.dower | set | messages: + msg331267 |
2018-12-07 05:09:57 | steve.dower | set | messages: + msg331266 |
2018-12-07 05:09:24 | steve.dower | set | messages: + msg331265 |
2018-11-29 00:24:21 | steve.dower | set | messages: + msg330643 |
2018-11-26 18:42:01 | steve.dower | set | messages: - msg330453 |
2018-11-26 18:41:44 | steve.dower | set | messages: + msg330453 |
2018-11-26 18:41:33 | steve.dower | set | messages: + msg330452 |
2018-11-23 19:57:16 | steve.dower | set | nosy:
+ terry.reedy messages: + msg330359 |
2018-11-23 14:56:34 | steve.dower | set | pull_requests: + pull_request9932 |
2018-11-21 00:29:52 | steve.dower | set | nosy:
+ vinay.sajip messages: + msg330166 |
2018-10-30 15:55:13 | steve.dower | set | keywords:
+ patch stage: patch review pull_requests: + pull_request9559 |
2018-10-20 15:19:38 | steve.dower | set | messages: + msg328147 |
2018-10-20 15:19:08 | steve.dower | set | messages: + msg328146 |
2018-10-14 06:02:43 | xtreak | set | nosy:
+ xtreak messages: + msg327691 |
2018-10-13 22:43:47 | steve.dower | create |