classification
Title: Decouple ensurepip from pip's internals using runpy
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.9, Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Marcus.Smith, brett.cannon, dstufft, hroncok, miss-islington, ncoghlan, ned.deily, paul.moore, pradyunsg, vstinner
Priority: normal Keywords: patch

Created on 2019-11-01 17:24 by pradyunsg, last changed 2020-03-10 21:40 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 17029 closed pradyunsg, 2019-11-01 17:25
PR 18901 merged hroncok, 2020-03-10 14:34
PR 18906 merged miss-islington, 2020-03-10 21:18
PR 18907 merged miss-islington, 2020-03-10 21:18
Messages (11)
msg355826 - (view) Author: Pradyun Gedam (pradyunsg) * Date: 2019-11-01 17:24
Inspired by https://github.com/python/cpython/pull/16782#discussion_r335285656

This changes ensurepip's current assumption that pip would expose a "main" function, which can be invoked to run pip.

I've selected all actively supported Python versions for this ticket, since this patch should be backported to all Python versions, to make it robust to internal changes within pip.
msg355828 - (view) Author: Pradyun Gedam (pradyunsg) * Date: 2019-11-01 17:38
Note that the patch uses the fact that:


```
def foo():
    try:
        raise Exception("returned")
    except Exception as e:
        print("except")
        return e.args[0]
    finally:
        print("finally")


print(foo())
```

will print:

```
except
finally
returned
```
msg356021 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-11-05 11:00
Can you explain how this would be rolled out?  Does this mean as of a planned release of pip the current ensurepip would break?  If so, when?  Is 2.7 still supported with that version of pip and, if so, have you tested the PR with it?
msg356031 - (view) Author: Pradyun Gedam (pradyunsg) * Date: 2019-11-05 13:05
On Tue, 5 Nov 2019 at 4:30 PM, Ned Deily <report@bugs.python.org> wrote:

>
> Ned Deily <nad@python.org> added the comment:
>
> Can you explain how this would be rolled out?

This, as in? If you're referring to this specific change, I don't think
there's anything special needed for this change (it's backwards compatible
with past pip releases, but that doesn't matter).

ensurepip only installs the bundled wheel it already has, into the
environment. Thus, existing ensurepip releases won't be broken.

However, when we bump up to pip 19.3 [1], we'd need to be updating the
invocation since it's using a pip internal and pip's internals can change
without notice even in a patch version.

This change ensures that ensurepip is robust to changes in the location of
pip's main function, thus making those updates as simple as a bundled wheel
change.

[1]: IIUC we try to keep CPython's bundled wheel in sync with latest)

Does this mean as of a planned release of pip the current ensurepip would
> break?

Yes. pip 19.3 made a change and as can be seen in the PR for that (sorry,
it's a GitHub PR and my internet is ridiculously slow right now).

  If so, when?

That's already been released. 🙃

Is 2.7 still supported with that version of pip and, if so, have you tested
> the PR with it?

Yes, pip will support 2.7 at least until CPython does (and likely longer).

>
> ----------
> nosy: +ned.deily
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue38662>
> _______________________________________
>
msg363846 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-10 21:16
New changeset 88f82b2b9ea3514359cb6e3218121f75334063ac by Miro Hrončok in branch 'master':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/88f82b2b9ea3514359cb6e3218121f75334063ac
msg363847 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-10 21:23
While it's not a major issue in CPython, I can say that being tightly coupled to pip internals is a major in Fedora which updates pip frequently and tries to support multiple pip versions.
msg363848 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-10 21:26
Miro Hrončok: "I wasn't even hoping for a backport. Thanks!" ( https://github.com/python/cpython/pull/18901#issuecomment-597324810 )

pip wheel files are still updated in stable 3.7 and 3.8 branches. Example in August 2018 in 3.7:

* commit c1c1a3396391dcc948d332607a0f673e4434da97
* bpo-37664
msg363849 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-10 21:29
> Yes, pip will support 2.7 at least until CPython does (and likely longer).

2.7 is no longer supported in CPython. So I will not backport the fix to 2.7. And I removed the 2.7 from versions of this issue.
msg363851 - (view) Author: miss-islington (miss-islington) Date: 2020-03-10 21:34
New changeset 8d5c958ddebbb96c2de5d58da4c7aa1790629cf9 by Miss Islington (bot) in branch '3.7':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/8d5c958ddebbb96c2de5d58da4c7aa1790629cf9
msg363852 - (view) Author: miss-islington (miss-islington) Date: 2020-03-10 21:37
New changeset 212acf544a41a983aea7166d43d8dfe9a8296655 by Miss Islington (bot) in branch '3.8':
bpo-38662: ensurepip invokes pip via runpy (GH-18901)
https://github.com/python/cpython/commit/212acf544a41a983aea7166d43d8dfe9a8296655
msg363854 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-03-10 21:40
Thanks Pradyun Gedam and Miro Hrončok for the fix.
History
Date User Action Args
2020-03-10 21:40:24vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg363854

stage: patch review -> resolved
2020-03-10 21:37:53miss-islingtonsetmessages: + msg363852
2020-03-10 21:34:54miss-islingtonsetmessages: + msg363851
2020-03-10 21:29:22vstinnersetmessages: + msg363849
2020-03-10 21:26:47vstinnersetmessages: + msg363848
2020-03-10 21:23:36vstinnersetmessages: + msg363847
2020-03-10 21:19:07vstinnersetcomponents: + Library (Lib)
versions: - Python 2.7, Python 3.5, Python 3.6
2020-03-10 21:18:30miss-islingtonsetpull_requests: + pull_request18262
2020-03-10 21:18:22miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request18261
2020-03-10 21:16:42vstinnersetnosy: + vstinner
messages: + msg363846
2020-03-10 14:34:06hroncoksetnosy: + hroncok
pull_requests: + pull_request18259
2019-11-05 13:05:26pradyunsgsetmessages: + msg356031
2019-11-05 11:00:27ned.deilysetnosy: + ned.deily
messages: + msg356021
2019-11-04 18:36:54brett.cannonsetnosy: + brett.cannon
2019-11-01 17:38:26pradyunsgsetmessages: + msg355828
2019-11-01 17:25:39pradyunsgsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16543
2019-11-01 17:24:24pradyunsgcreate