Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support -3.11-arm64 in py.exe launcher #90724

Closed
zooba opened this issue Jan 28, 2022 · 16 comments
Closed

Support -3.11-arm64 in py.exe launcher #90724

zooba opened this issue Jan 28, 2022 · 16 comments
Assignees
Labels
3.11 bug and security fixes OS-windows type-feature A feature request or enhancement

Comments

@zooba
Copy link
Member

zooba commented Jan 28, 2022

BPO 46566
Nosy @brettcannon, @pfmoore, @tjguk, @zware, @zooba, @tommyvct, @conioh
PRs
  • bpo-46566: Add new py.exe launcher implementation #32062
  • bpo-46566: Make test_launcher more robust to a variety of installs #32204
  • Files
  • launcher.msi
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/zooba'
    closed_at = <Date 2022-03-28.23:22:12.800>
    created_at = <Date 2022-01-28.16:52:59.083>
    labels = ['type-feature', 'OS-windows', '3.11']
    title = 'Support -3.11-arm64 in py.exe launcher'
    updated_at = <Date 2022-03-30.21:18:48.191>
    user = 'https://github.com/zooba'

    bugs.python.org fields:

    activity = <Date 2022-03-30.21:18:48.191>
    actor = 'steve.dower'
    assignee = 'steve.dower'
    closed = True
    closed_date = <Date 2022-03-28.23:22:12.800>
    closer = 'steve.dower'
    components = ['Windows']
    creation = <Date 2022-01-28.16:52:59.083>
    creator = 'steve.dower'
    dependencies = []
    files = ['50701']
    hgrepos = []
    issue_num = 46566
    keywords = ['patch']
    message_count = 16.0
    messages = ['412008', '412481', '414299', '414928', '414929', '414936', '415165', '415168', '415173', '415178', '415181', '415844', '416009', '416228', '416229', '416399']
    nosy_count = 7.0
    nosy_names = ['brett.cannon', 'paul.moore', 'tim.golden', 'zach.ware', 'steve.dower', 'TommyVCT', 'conio']
    pr_nums = ['32062', '32204']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue46566'
    versions = ['Python 3.11']

    @zooba
    Copy link
    Member Author

    zooba commented Jan 28, 2022

    We now have a traditional installer that runs on Windows ARM64 devices, but it omits the py.exe launcher. This is because sys.winver is "3.11-arm64" and hence the registry tag (see PEP-514) is not detected by the launcher.

    Currently, the launcher is not included in the installer and appears as disabled with an explanatory note.

    We need to add support for it eventually, and possibly block the final/RC/beta releases if we don't have it. (We don't ship the launcher with Nuget or Store packages, so nobody will miss it there, but they probably will here.)

    32-bit apps can still run on ARM64, so it isn't essential to make it a native process. It just has to be able to detect the registry keys.

    It should still support 32-bit and 64-bit installs, because those can also be run on ARM64 devices (though there are numerous things that don't work as well as a native build).

    Possibly we want to just do the work to directly follow PEP-514 now? Depends on who volunteers to work on this, I guess.

    @zooba zooba added 3.11 bug and security fixes OS-windows type-feature A feature request or enhancement labels Jan 28, 2022
    @tommyvct
    Copy link
    Mannequin

    tommyvct mannequin commented Feb 3, 2022

    PEP-514 looks good to me

    @zooba
    Copy link
    Member Author

    zooba commented Mar 1, 2022

    I'm working on this now.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 11, 2022

    I'm working on what's become a rewrite of the launcher. If anyone would like to follow along, you can see my changes at main...zooba:bpo-46566

    It's still missing some functionality, and I'm not sure it's any faster or less complex than the previous version :) But it should handle ARM64 properly, and also knows how to do an install of a requested version if you don't already have it (still TBD whether that stays)

    @zooba
    Copy link
    Member Author

    zooba commented Mar 11, 2022

    Hah, that's funny URL formatting. Let's see if this is any better:

    <main...zooba:bpo-46566\>

    @zware
    Copy link
    Member

    zware commented Mar 11, 2022

    If it's already turning into a rewrite, how feasible would it be to adopt Brett's py launcher?

    @zooba
    Copy link
    Member Author

    zooba commented Mar 14, 2022

    If it's already turning into a rewrite, how feasible would it be to adopt Brett's py launcher?

    I looked at it already, and I'd have to write literally the same code to implement what's needed :) (as well as learning Rust and convincing everyone to let us use Rust in CPython...). Practically nothing can be reused - there's no registry, no shebang handling, and our process launching on Windows is already very complex (and has to remain that way for compatibility).

    For now, the old launcher will remain to be used for venv redirectors, but I've got the setup in the new one to be able to play the same role, as well as potentially being able to be a script or .pyz launcher with a simple rename.

    If Brett's proposal for extensions (other executables on PATH that know how to identify Python installs) happens, we'll probably copy it, though PEP-514 covers Windows adequately (and the rewrite allows non-PythonCore installs to be found/launched). But the codebase itself isn't helpful.

    @pfmoore
    Copy link
    Member

    pfmoore commented Mar 14, 2022

    as well as potentially being able to be a script or .pyz launcher with a simple rename.

    Would it be possible to also make the launcher work when prepended to a zipfile? That's a really useful use-case (make a zipapp automatically runnable, but still a single file) that at the moment needs a 3rd party launcher (Vinay's simple-launcher project).

    If not, then that's fine, but if we're already doing a significant rewrite that might be a good time to add it.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 14, 2022

    I'd like to, the main challenge with that is it'd invalidate the code signature on the file, which will make it basically unusable (at the very least, you'll get warnings). A simple rename does not.

    But yeah, it can probably go in. Hopefully my restructure will make it easier to follow where certain things get checked.

    @brettcannon
    Copy link
    Member

    "Practically nothing can be reused - there's no registry, no shebang handling, and our process launching on Windows is already very complex (and has to remain that way for compatibility)."

    I do process the shebang to restrict searching, or did you mean something else?

    And registry support is planned.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 14, 2022

    I do process the shebang to restrict searching, or did you mean something else?

    That's what I meant. Guess I missed seeing it when scanning the code (probably I should've read the docs :D )

    And registry support is planned.

    It's attached to a milestone with no due date and 0% completion ;) Being aware that you'll need to do it isn't the same as planning to do it.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 23, 2022

    Posted my WIP as a draft PR. I'm still adding docs, tests and ironing out minor issues, but the core functionality is there (as well as some added bonuses). Feel free to review, and importantly to mark anywhere you'd like more explanatory comments.

    Currently the main known issue is that the "-3-32" (and now-deprecated "-3-64") arguments are not supported. But they will be! There are also likely to be a range of shebang templates that need better handling, and I'll go through the open issues to see what ought to be fixed there compared to today's version.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 25, 2022

    I think this PR is "ready enough" to get in for broader testing, but in case anyone wants to try it first, I've attached a build from my own machine.

    You may need Programs & Features to remove any existing "Python Launcher" before installing this MSI, and you will need to do that to remove this later on, but it should otherwise behave identically to a per-user installed launcher.

    @zooba
    Copy link
    Member Author

    zooba commented Mar 28, 2022

    New changeset bad86a6 by Steve Dower in branch 'main':
    bpo-46566: Add new py.exe launcher implementation (GH-32062)
    bad86a6

    @zooba
    Copy link
    Member Author

    zooba commented Mar 28, 2022

    It's in! Any new issues probably won't be found until the next release, so I'll close this and we can open new bugs.

    @zooba zooba closed this as completed Mar 28, 2022
    @zooba zooba closed this as completed Mar 28, 2022
    @zooba
    Copy link
    Member Author

    zooba commented Mar 30, 2022

    New changeset f3d5715 by Steve Dower in branch 'main':
    bpo-46566: Make test_launcher more robust to a variety of installs (GH-32204)
    f3d5715

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 bug and security fixes OS-windows type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants