Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple ensurepip from pip's internals using runpy #82843

Closed
pradyunsg opened this issue Nov 1, 2019 · 11 comments
Closed

Decouple ensurepip from pip's internals using runpy #82843

pradyunsg opened this issue Nov 1, 2019 · 11 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir

Comments

@pradyunsg
Copy link
Member

BPO 38662
Nosy @brettcannon, @pfmoore, @ncoghlan, @vstinner, @ned-deily, @dstufft, @pradyunsg, @hroncok, @miss-islington
PRs
  • bpo-38662: Invoke pip via runpy, in ensurepip #17029
  • bpo-38662: Invoke pip via runpy, in ensurepip #18901
  • [3.8] bpo-38662: ensurepip invokes pip via runpy (GH-18901) #18906
  • [3.7] bpo-38662: ensurepip invokes pip via runpy (GH-18901) #18907
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-03-10.21:40:24.083>
    created_at = <Date 2019-11-01.17:24:24.199>
    labels = ['3.8', '3.7', 'library', '3.9']
    title = "Decouple ensurepip from pip's internals using runpy"
    updated_at = <Date 2020-03-10.21:40:24.083>
    user = 'https://github.com/pradyunsg'

    bugs.python.org fields:

    activity = <Date 2020-03-10.21:40:24.083>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-03-10.21:40:24.083>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2019-11-01.17:24:24.199>
    creator = 'pradyunsg'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38662
    keywords = ['patch']
    message_count = 11.0
    messages = ['355826', '355828', '356021', '356031', '363846', '363847', '363848', '363849', '363851', '363852', '363854']
    nosy_count = 10.0
    nosy_names = ['brett.cannon', 'paul.moore', 'ncoghlan', 'vstinner', 'ned.deily', 'dstufft', 'pradyunsg', 'Marcus.Smith', 'hroncok', 'miss-islington']
    pr_nums = ['17029', '18901', '18906', '18907']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue38662'
    versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

    @pradyunsg
    Copy link
    Member Author

    Inspired by #16782 (comment)

    This changes ensurepip's current assumption that pip would expose a "main" function, which can be invoked to run pip.

    I've selected all actively supported Python versions for this ticket, since this patch should be backported to all Python versions, to make it robust to internal changes within pip.

    @pradyunsg pradyunsg added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Nov 1, 2019
    @pradyunsg
    Copy link
    Member Author

    Note that the patch uses the fact that:

    def foo():
        try:
            raise Exception("returned")
        except Exception as e:
            print("except")
            return e.args[0]
        finally:
            print("finally")
    
    
    print(foo())
    

    will print:

    except
    finally
    returned
    

    @ned-deily
    Copy link
    Member

    Can you explain how this would be rolled out? Does this mean as of a planned release of pip the current ensurepip would break? If so, when? Is 2.7 still supported with that version of pip and, if so, have you tested the PR with it?

    @pradyunsg
    Copy link
    Member Author

    On Tue, 5 Nov 2019 at 4:30 PM, Ned Deily <report@bugs.python.org> wrote:

    Ned Deily <nad@python.org> added the comment:

    Can you explain how this would be rolled out?

    This, as in? If you're referring to this specific change, I don't think
    there's anything special needed for this change (it's backwards compatible
    with past pip releases, but that doesn't matter).

    ensurepip only installs the bundled wheel it already has, into the
    environment. Thus, existing ensurepip releases won't be broken.

    However, when we bump up to pip 19.3 [1], we'd need to be updating the
    invocation since it's using a pip internal and pip's internals can change
    without notice even in a patch version.

    This change ensures that ensurepip is robust to changes in the location of
    pip's main function, thus making those updates as simple as a bundled wheel
    change.

    [1]: IIUC we try to keep CPython's bundled wheel in sync with latest)

    Does this mean as of a planned release of pip the current ensurepip would

    break?

    Yes. pip 19.3 made a change and as can be seen in the PR for that (sorry,
    it's a GitHub PR and my internet is ridiculously slow right now).

    If so, when?

    That's already been released. 🙃

    Is 2.7 still supported with that version of pip and, if so, have you tested

    the PR with it?

    Yes, pip will support 2.7 at least until CPython does (and likely longer).

    ----------
    nosy: +ned.deily


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue38662\>


    @vstinner
    Copy link
    Member

    New changeset 88f82b2 by Miro Hrončok in branch 'master':
    bpo-38662: ensurepip invokes pip via runpy (GH-18901)
    88f82b2

    @vstinner vstinner added the stdlib Python modules in the Lib dir label Mar 10, 2020
    @vstinner
    Copy link
    Member

    While it's not a major issue in CPython, I can say that being tightly coupled to pip internals is a major in Fedora which updates pip frequently and tries to support multiple pip versions.

    @vstinner
    Copy link
    Member

    Miro Hrončok: "I wasn't even hoping for a backport. Thanks!" ( #18901 (comment) )

    pip wheel files are still updated in stable 3.7 and 3.8 branches. Example in August 2018 in 3.7:

    @vstinner
    Copy link
    Member

    Yes, pip will support 2.7 at least until CPython does (and likely longer).

    2.7 is no longer supported in CPython. So I will not backport the fix to 2.7. And I removed the 2.7 from versions of this issue.

    @miss-islington
    Copy link
    Contributor

    New changeset 8d5c958 by Miss Islington (bot) in branch '3.7':
    bpo-38662: ensurepip invokes pip via runpy (GH-18901)
    8d5c958

    @miss-islington
    Copy link
    Contributor

    New changeset 212acf5 by Miss Islington (bot) in branch '3.8':
    bpo-38662: ensurepip invokes pip via runpy (GH-18901)
    212acf5

    @vstinner
    Copy link
    Member

    Thanks Pradyun Gedam and Miro Hrončok for the fix.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants