classification
Title: In ./Doc, "make html" and "make build" should depend on "make venv"
Type: enhancement Stage: resolved
Components: Demos and Tools, Documentation Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, hroncok, jack__d, lukasz.langa, miss-islington, ned.deily, pablogsal, petr.viktorin
Priority: release blocker Keywords: patch

Created on 2021-07-28 04:47 by jack__d, last changed 2021-08-30 21:56 by lukasz.langa. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 27403 merged jack__d, 2021-07-28 04:57
PR 27410 merged miss-islington, 2021-07-28 13:13
PR 27411 merged miss-islington, 2021-07-28 13:13
PR 27635 merged lukasz.langa, 2021-08-06 17:40
PR 27636 merged miss-islington, 2021-08-06 18:14
PR 27637 merged miss-islington, 2021-08-06 18:18
PR 28075 merged lukasz.langa, 2021-08-30 21:01
Messages (19)
msg398344 - (view) Author: Jack DeVries (jack__d) * Date: 2021-07-28 04:47
In Doc/Makefile, all of the build rules should be dependent on the existence of a virtual environment. I could see this being controversial, because folks who have these tools installed elsewhere might prefer not to have a venv made for them, but my personal workflow is to strive to keep my system site-packages folder as empty as possible and to use virtual environments frequently. In any case, I'm interested to hear what we think.

Momentarily, I will attach a PR where I went ahead and made these changes. Here is a summary of the changes:

- venv rule is now conditional, and only does anything if $VENVDIR does 
  not exist
- add rule "clean-venv". This can be removed – what do you all think?
- build rule is dependent on venv
- as a result, rules dependent on build are dependent on venv
  - html
  - latex
  - etc.
- update Doc/README.rst appropriately. Now users only need to type 
  ``make html`` – nice!

Let me know what you think. I may have a blind spot to others' workflows! Also, I'm not a Windows user, so I have no insight as to whether ``make.bat`` needs to be updated as well.
msg398373 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-28 13:13
New changeset d22c876d5ac5fa464337d2e82654b8d87a83cb1b by Jack DeVries in branch 'main':
bpo-44756: in ./Doc, `make build` depends on `make html` (#27403)
https://github.com/python/cpython/commit/d22c876d5ac5fa464337d2e82654b8d87a83cb1b
msg398381 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-28 13:35
New changeset f113195ac85d58bdde348b5bb74eb97e6485e408 by Miss Islington (bot) in branch '3.10':
bpo-44756: in ./Doc, `make build` depends on `make html` (GH-27403) (GH-27410)
https://github.com/python/cpython/commit/f113195ac85d58bdde348b5bb74eb97e6485e408
msg398383 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-28 13:40
New changeset fd2c2465d49a8bcb32eeda82d694eed48e57ba79 by Miss Islington (bot) in branch '3.9':
bpo-44756: in ./Doc, `make build` depends on `make html` (GH-27403) (GH-27411)
https://github.com/python/cpython/commit/fd2c2465d49a8bcb32eeda82d694eed48e57ba79
msg398384 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-07-28 13:41
Thanks, Jack! ✨ 🍰 ✨
msg398827 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-08-03 13:54
I'm one of those who disagree with "make html" suddenly downloading code from the internet and running it, but I guess I'm in a minority. I'll switch to using `sphinx-build` directly.

Still, I'd have appreciated a heads-up notice.
msg398840 - (view) Author: Jack DeVries (jack__d) * Date: 2021-08-03 17:11
@petr.viktorin a whatsnew entry was added, what more notice could have been provided?

I have an idea for an alternative that might be better. What if ``make clean`` deletes and restores the venv only if it already existed in the first place?
msg398848 - (view) Author: Jack DeVries (jack__d) * Date: 2021-08-03 17:57
Actually, I tested out that idea (bpo-44756-doc-make">https://github.com/python/cpython/compare/main...jdevries3133:bpo-44756-doc-make), and I don't think its as nice of a solution. I think it is valuable for new contributors to be able to type "make html" and have it work. Look at how much the change simplifies the README for new contributors to setup their documentation build toolchain. Then, look at how many ``docs@python`` open issues there are. We need documentation contributors, and there is value in simplifying the process for them.

Additionally, this is the extent of the "downloading code from the internet and running it" that occurs::

    pip install -U pip setuptools
    pip install sphynx blurb python-docs-theme

If running that is ever unsafe, we have big problems!
msg398868 - (view) Author: Miro Hrončok (hroncok) * Date: 2021-08-04 08:53
The 3.10.0rc1 source tarballs contain the Docs/venv directory populated with pablogsal's venv: issue44824
msg398874 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-08-04 09:21
The issue this (or lack of communication about it) caused in rc1 is tracked in https://bugs.python.org/issue44823

> @petr.viktorin a whatsnew entry was added, what more notice could have been provided?

Ideally, the python-dev mailing list (or Discourse).

> pip install sphinx blurb python-docs-theme
> If running that is ever unsafe, we have big problems!

Who is "we"?
We do have big problems. Anyone who can upload wheels for sphinx blurb python-docs-theme or any of their dependencies (or anyone who has their credentials) can now easily put code on machines of CPython developers.

For example, PyPI doesn't guarantee that wheels correspond to sources. "Markupsafe" is particularly dangerous because the wheels are platform-specific and have compiled code, so tampering is nearly undetectable. (But if another dependency starts using platform-specific wheels, I don't think anyone would notice.)
msg398875 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2021-08-04 09:29
I've sent a mail to python-dev: https://mail.python.org/archives/list/python-dev@python.org/thread/MGPNI7OSA7UXNOTVDVW2I2GUMXV25FRS/
msg398884 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2021-08-04 10:13
Note this issue has impacted negatively the release process of 3.10.0rc1:

https://bugs.python.org/issue44823
msg399101 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2021-08-06 16:31
There was a previous attempt at adding a dependency on venv that ended up being reverted (Issue30487 and 122fc136b34e11906466851e77bb6959946467ee0). Over the years we have had a number of iterations of tweaking Doc/Makefile to balance ease of use (by providing the venv) with those of more advanced users who build docs as part of their automated processeses. I think what we had prior to this most recent change worked reasonably well for all. The previous process is clearly documented, for example in the Dev Guide (https://devguide.python.org/documenting/?highlight=venv#building-the-documentation) and does not strike me as very onerous. I appreciate the motivation behind this change but I really think there isn't a problem here that needs to be solved and I would support reverting the change for 3.9 and 3.10 and *perhaps* reconsider something for 3.11.
msg399116 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-06 18:14
New changeset 55fa87b1ef46bbb9db7612b2dc2dd7fb039d9bc3 by Łukasz Langa in branch 'main':
bpo-44756: [docs] revert automated virtual environment creation on `make html` (GH-27635)
https://github.com/python/cpython/commit/55fa87b1ef46bbb9db7612b2dc2dd7fb039d9bc3
msg399124 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-06 18:42
New changeset 91f6d386691c060dfaa870d74b370e9c10eb8cd8 by Miss Islington (bot) in branch '3.9':
bpo-44756: [docs] revert automated virtual environment creation on `make html` (GH-27635) (GH-27636)
https://github.com/python/cpython/commit/91f6d386691c060dfaa870d74b370e9c10eb8cd8
msg399130 - (view) Author: miss-islington (miss-islington) Date: 2021-08-06 19:16
New changeset 1e9c9ca570ffe92dc6a60825c2f9c91865129b8f by Miss Islington (bot) in branch '3.10':
bpo-44756: [docs] revert automated virtual environment creation on `make html` (GH-27635)
https://github.com/python/cpython/commit/1e9c9ca570ffe92dc6a60825c2f9c91865129b8f
msg399135 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-06 19:53
Revert done. Re-closing!
msg400672 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-30 21:54
New changeset 5246dbc2a12bf8e64e18efee2fdce02a350bbf09 by Łukasz Langa in branch 'main':
bpo-44756: Remove misleading NEWS entries of a change that was reverted before release (GH-28075)
https://github.com/python/cpython/commit/5246dbc2a12bf8e64e18efee2fdce02a350bbf09
msg400673 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-30 21:56
New changeset 9ef1843892500e209afe5d916db26ecdbccd5e62 by Łukasz Langa in branch '3.9':
bpo-44756: Remove misleading NEWS entries of a change that was reverted before release (GH-28075)
https://github.com/python/cpython/commit/9ef1843892500e209afe5d916db26ecdbccd5e62
History
Date User Action Args
2021-08-30 21:56:40lukasz.langasetmessages: + msg400673
2021-08-30 21:54:57lukasz.langasetmessages: + msg400672
2021-08-30 21:01:17lukasz.langasetpull_requests: + pull_request26519
2021-08-06 19:53:19lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg399135

stage: patch review -> resolved
2021-08-06 19:16:02miss-islingtonsetmessages: + msg399130
2021-08-06 18:42:35lukasz.langasetmessages: + msg399124
2021-08-06 18:18:53miss-islingtonsetpull_requests: + pull_request26131
2021-08-06 18:14:09miss-islingtonsetpull_requests: + pull_request26130
2021-08-06 18:14:07lukasz.langasetmessages: + msg399116
2021-08-06 17:40:46lukasz.langasetstage: needs patch -> patch review
pull_requests: + pull_request26129
2021-08-06 16:34:07ned.deilysetversions: + Python 3.9, Python 3.10
2021-08-06 16:31:08ned.deilysetstatus: closed -> open
priority: normal -> release blocker


nosy: + ned.deily
messages: + msg399101
resolution: fixed -> (no value)
stage: resolved -> needs patch
2021-08-04 10:13:04pablogsalsetmessages: + msg398884
2021-08-04 09:59:41pablogsalsetnosy: + pablogsal
2021-08-04 09:29:43petr.viktorinsetmessages: + msg398875
2021-08-04 09:21:58petr.viktorinsetmessages: + msg398874
2021-08-04 08:53:40hroncoksetnosy: + hroncok
messages: + msg398868
2021-08-03 17:57:31jack__dsetmessages: + msg398848
2021-08-03 17:11:57jack__dsetmessages: + msg398840
2021-08-03 13:54:19petr.viktorinsetnosy: + petr.viktorin
messages: + msg398827
2021-07-28 13:41:14lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg398384

stage: patch review -> resolved
2021-07-28 13:40:27lukasz.langasetmessages: + msg398383
2021-07-28 13:35:26lukasz.langasetmessages: + msg398381
2021-07-28 13:13:42miss-islingtonsetpull_requests: + pull_request25944
2021-07-28 13:13:37miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request25943
2021-07-28 13:13:34lukasz.langasetnosy: + lukasz.langa
messages: + msg398373
2021-07-28 04:57:31jack__dsetkeywords: + patch
stage: patch review
pull_requests: + pull_request25936
2021-07-28 04:47:25jack__dcreate