Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin Jinja2 to fix docs build #91294

Closed
hugovk opened this issue Mar 28, 2022 · 17 comments
Closed

Pin Jinja2 to fix docs build #91294

hugovk opened this issue Mar 28, 2022 · 17 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir

Comments

@hugovk
Copy link
Member

hugovk commented Mar 28, 2022

BPO 47138
Nosy @ned-deily, @ambv, @zooba, @hugovk, @miss-islington, @m-aciek
PRs
  • [3.7] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32111
  • [3.8] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32109
  • [3.9] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32153
  • [3.10] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 #32154
  • bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms #32161
  • [3.10] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32163
  • [3.8] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32167
  • [3.7] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32168
  • [3.9] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32169
  • [3.7] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms #32182
  • [3.8] bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161) #32183
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2022-03-28.08:17:25.467>
    labels = ['3.7', '3.8', '3.9', '3.10', 'docs']
    title = 'Pin Jinja2 to fix docs build'
    updated_at = <Date 2022-04-08.13:06:24.310>
    user = 'https://github.com/hugovk'

    bugs.python.org fields:

    activity = <Date 2022-04-08.13:06:24.310>
    actor = 'lukasz.langa'
    assignee = 'docs@python'
    closed = False
    closed_date = None
    closer = None
    components = ['Documentation']
    creation = <Date 2022-03-28.08:17:25.467>
    creator = 'hugovk'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 47138
    keywords = ['patch']
    message_count = 14.0
    messages = ['416149', '416151', '416158', '416167', '416171', '416194', '416195', '416196', '416197', '416209', '416233', '416235', '416285', '416983']
    nosy_count = 7.0
    nosy_names = ['ned.deily', 'docs@python', 'lukasz.langa', 'steve.dower', 'hugovk', 'miss-islington', 'Maciej Olko']
    pr_nums = ['32111', '32109', '32153', '32154', '32161', '32163', '32167', '32168', '32169', '32182', '32183']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue47138'
    versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

    @hugovk
    Copy link
    Member Author

    hugovk commented Mar 28, 2022

    The docs build on GitHub Actions is passing on main but has started failing for the 3.7 - 3.10 branches:

    Missing the required blurb or sphinx-build tools.
    Please run 'make venv' to install local copies.
    
    make[1]: *** [Makefile:50: build] Error 1
    

    https://github.com/python/cpython/actions/workflows/doc.yml
    https://github.com/python/cpython/runs/5714593700?check_suite_focus=true

    This is because the PATH=./venv/bin:$PATH sphinx-build --version check is failing:

    /home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/util/docutils.py:45: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      __version_info__ = tuple(LooseVersion(docutils.__version__).version)
    Traceback (most recent call last):
      File "/home/runner/work/cpython/cpython/Doc/./venv/bin/sphinx-build", line 5, in <module>
        from sphinx.cmd.build import main
      File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/cmd/build.py", line 25, in <module>
        from sphinx.application import Sphinx
      File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/application.py", line 42, in <module>
        from sphinx.registry import SphinxComponentRegistry
      File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/registry.py", line 24, in <module>
        from sphinx.builders import Builder
      File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 26, in <module>
        from sphinx.util import import_object, logging, rst, progress_message, status_iterator
      File "/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/sphinx/util/rst.py", line 22, in <module>
        from jinja2 import environmentfilter
    ImportError: cannot import name 'environmentfilter' from 'jinja2' (/home/runner/work/cpython/cpython/Doc/venv/lib/python3.10/site-packages/jinja2/__init__.py)
    

    This is because 3.10 and 3.9 are still using old versions of Sphinx (3.2.1 and 2.4.4 respectively) which is incompatible with the newest Jinja2:

    pallets/jinja#1630

    main is using a newer Sphinx (4.2.0), so it passes.

    Bumping to Sphinx 4.5.0 will fix both 3.9 and 3.10. 3.7 and 3.8 are affected too. We can bump main to use 4.5.0 as well.

    @hugovk hugovk added 3.11 only security fixes 3.10 only security fixes 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Mar 28, 2022
    @hugovk hugovk added 3.10 only security fixes docs Documentation in the Doc dir 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes labels Mar 28, 2022
    @hugovk hugovk added the docs Documentation in the Doc dir label Mar 28, 2022
    @hugovk
    Copy link
    Member Author

    hugovk commented Mar 28, 2022

    re: https://bugs.python.org/issue45618#msg416017

    Maciej Olko, please could you update these PRs to keep the Sphinx version the same and only pin Jinja2 to match?

    3.7: #32109
    3.8: #32111

    I guess update them to use this bpo too. Thanks!

    @hugovk hugovk changed the title Bump Sphinx to fix docs build Bump Sphinx or pin Jinja2 to fix docs build Mar 28, 2022
    @hugovk hugovk changed the title Bump Sphinx to fix docs build Bump Sphinx or pin Jinja2 to fix docs build Mar 28, 2022
    @m-aciek
    Copy link
    Mannequin

    m-aciek mannequin commented Mar 28, 2022

    I've update both PRs to contain the pin only.

    @ned-deily
    Copy link
    Member

    Thank you both for addressing this. The main reason for minimizing changes to the docs build chain for brances in bugfix or security-fix-only phases is that many distributors of Python also release the Python docs as a package and they do not necessarily rely on using our fairly-recent venv step in the Docs Makefile. In particular, they want to use the versions of Sphinx et al that they release as packages. If we change the required versions for these mid-stream, they not only have to update the docs themselves but have to go through the process of updating the other packages (like Sphinx) which may have other consequences. This can also impact our own on-line doc building process. (Version updates for a new Python feature release prior to its entering beta phase are not a problem.) So it is best to do the minimal amount of version updating for releases in the field, preferably none (as in pinning existing versions). In that spirit, we should also just pin the Jinja2 version for 3.10 and 3.9.

    @hugovk
    Copy link
    Member Author

    hugovk commented Mar 28, 2022

    I cherry picked Maciej's 3.8 commit into 3.9 and 3.10 PRs:

    3.9: #32153
    3.10: #32154

    And marking this issue as not relevant to 3.11; instead I bumped the Sphinx version in bpo-47126 / #76305 as it's directly relevant there.

    Thanks!

    @hugovk hugovk removed the 3.11 only security fixes label Mar 28, 2022
    @hugovk hugovk changed the title Bump Sphinx or pin Jinja2 to fix docs build Pin Jinja2 to fix docs build Mar 28, 2022
    @hugovk hugovk removed the 3.11 only security fixes label Mar 28, 2022
    @hugovk hugovk changed the title Bump Sphinx or pin Jinja2 to fix docs build Pin Jinja2 to fix docs build Mar 28, 2022
    @ned-deily
    Copy link
    Member

    New changeset 25f00bf by m-aciek in branch '3.7':
    bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32111)
    25f00bf

    @ned-deily
    Copy link
    Member

    New changeset 9194a7b by Hugo van Kemenade in branch '3.10':
    bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32154)
    9194a7b

    @ned-deily
    Copy link
    Member

    New changeset 25c40bd by Hugo van Kemenade in branch '3.9':
    bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32153)
    25c40bd

    @ned-deily
    Copy link
    Member

    Thanks, everyone! Merged for 3.10.5, 3.9.13, and 3.7.14. I'll leave the issue in pending state until merged for 3.8.14.

    @zooba
    Copy link
    Member

    zooba commented Mar 28, 2022

    New changeset 295114d by Steve Dower in branch 'main':
    bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
    295114d

    @zooba
    Copy link
    Member

    zooba commented Mar 28, 2022

    New changeset 0dfabf9 by Steve Dower in branch '3.10':
    bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
    0dfabf9

    @miss-islington
    Copy link
    Contributor

    New changeset d2c6a44 by Miss Islington (bot) in branch '3.9':
    bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32161)
    d2c6a44

    @ned-deily
    Copy link
    Member

    New changeset d97497b by Steve Dower in branch '3.7':
    bpo-47138: Ensure Windows docs build uses the same pinned version as other platforms (GH-32182)
    d97497b

    @ambv
    Copy link
    Contributor

    ambv commented Apr 8, 2022

    New changeset d35af52 by m-aciek in branch '3.8':
    [3.8] bpo-47138: Fix documentation build by pinning Jinja version to 3.0.3 (GH-32109)
    d35af52

    @kumaraditya303
    Copy link
    Contributor

    @hugovk Can this be closed now ?

    @hugovk
    Copy link
    Member Author

    hugovk commented May 9, 2022

    We're still waiting for #32183 to be merged for 3.8.

    Ping 3.8 RM @ambv.

    @hugovk
    Copy link
    Member Author

    hugovk commented May 9, 2022

    #32183 is now merged! Closing, thanks all!

    @hugovk hugovk closed this as completed May 9, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants