Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Documentation builds fail with Sphinx 3.2.1 #89781

Closed
m-aciek mannequin opened this issue Oct 26, 2021 · 16 comments
Closed

Documentation builds fail with Sphinx 3.2.1 #89781

m-aciek mannequin opened this issue Oct 26, 2021 · 16 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir release-blocker type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@m-aciek
Copy link
Mannequin

m-aciek mannequin commented Oct 26, 2021

BPO 45618
Nosy @ned-deily, @ambv, @JulienPalard, @pablogsal, @hugovk, @miss-islington, @m-aciek, @erlend-aasland
PRs
  • bpo-45618: Fix documentation build by pinning Docutils version #29230
  • [3.10] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) #29240
  • [3.9] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) #29241
  • [3.8] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) (GH-29241) #29245
  • bpo-45618: Update Sphinx version used to build the documentation to 4.2.0 #29256
  • [3.7] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 #31476
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-03-28.13:55:34.977>
created_at = <Date 2021-10-26.22:34:59.629>
labels = ['3.8', '3.9', '3.10', 'release-blocker', '3.7', 'type-crash', 'docs']
title = 'Documentation builds fail with Sphinx 3.2.1'
updated_at = <Date 2022-03-28.13:55:34.976>
user = 'https://github.com/m-aciek'

    bugs.python.org fields:

    activity = <Date 2022-03-28.13:55:34.976>
actor = 'ned.deily'
assignee = 'docs@python'
closed = True
closed_date = <Date 2022-03-28.13:55:34.977>
closer = 'ned.deily'
components = ['Documentation']
creation = <Date 2021-10-26.22:34:59.629>
creator = 'Maciej Olko'
dependencies = []
files = []
hgrepos = []
issue_num = 45618
keywords = ['patch']
message_count = 16.0
messages = ['405060', '405061', '405099', '405101', '405102', '405118', '405125', '405134', '405135', '405229', '405420', '413665', '415979', '416017', '416148', '416150']
nosy_count = 9.0
nosy_names = ['ned.deily', 'docs@python', 'lukasz.langa', 'mdk', 'pablogsal', 'hugovk', 'miss-islington', 'Maciej Olko', 'erlendaasland']
pr_nums = ['29230', '29240', '29241', '29245', '29256', '31476']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue45618'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']
    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Oct 26, 2021

    Since the release of Docutils 0.18 today building Python documentation fails.

    In CPython repository we have Sphinx pinned to 3.2.1, which has too loose requirement for Docutils version in install_requires – docutils>=0.12.

    Example of failing build: https://github.com/python/python-docs-pl/runs/4009459257.

    My suggestion is to pin down docutils in docs requirements.

    @m-aciek m-aciek mannequin added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.11 only security fixes 3.10 only security fixes labels Oct 26, 2021
    @m-aciek m-aciek mannequin assigned docspython Oct 26, 2021
    @m-aciek m-aciek mannequin added docs Documentation in the Doc dir 3.7 (EOL) end of life type-crash A hard crash of the interpreter, possibly with a core dump 3.8 only security fixes 3.9 only security fixes 3.11 only security fixes 3.10 only security fixes labels Oct 26, 2021
    @m-aciek m-aciek mannequin assigned docspython Oct 26, 2021
    @m-aciek m-aciek mannequin added docs Documentation in the Doc dir type-crash A hard crash of the interpreter, possibly with a core dump 3.11 only security fixes and removed 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.11 only security fixes labels Oct 26, 2021
    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Oct 26, 2021

    The pin can be removed after bumping Sphinx to at least 3.5.4, which introduced upper limit for docutils version.

    sphinx-doc/sphinx@9263eea

    @ned-deily ned-deily added 3.9 only security fixes release-blocker labels Oct 27, 2021
    @ned-deily ned-deily added 3.9 only security fixes release-blocker labels Oct 27, 2021
    @ned-deily
    Copy link
    Member

    New changeset bcee6aa by m-aciek in branch 'main':
    bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230)
    bcee6aa

    @ned-deily
    Copy link
    Member

    New changeset 2b7b7c7 by Miss Islington (bot) in branch '3.10':
    bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) (GH-29240)
    2b7b7c7

    @ned-deily
    Copy link
    Member

    New changeset 30c1f18 by Ned Deily in branch '3.9':
    [3.9] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) (GH-29241)
    30c1f18

    @ned-deily
    Copy link
    Member

    All of the open branches are affected by this since we also have to build documentation for security-fix-only branches when releases. PRs for 3.7 and 3.6 forthcoming.

    @ned-deily ned-deily added 3.7 (EOL) end of life 3.8 only security fixes labels Oct 27, 2021
    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Oct 27, 2021

    For what it's worth I have just successfully built documentation on 3.7 and 3.6 branches locally. It looks like Sphinx 2.3.1 used there doesn't use this part of docutils API that have changed recently.

    Sphinx 2.3.1 has requirement of "docutils>=0.12", just like 3.2.1.

    We can pin docutils version to current (0.18) "just in case" to prevent a breakage in the future or do nothing – maybe it won't break – maybe the docutils API used is narrow enough.

    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Oct 27, 2021

    It occurs that the documentation builds without warnings on most recent Sphinx -- version 4.2.0. Probably we should bump the version in 3.11, 3.10 and 3.9 to 4.2.0. That would enable us to remove the pin on docutils.

    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Oct 27, 2021

    I'm sorry for the confusion. I might have make more checking on my side before proposing a fix.

    @ambv
    Copy link
    Contributor

    ambv commented Oct 28, 2021

    New changeset f240714 by Ned Deily in branch '3.8':
    [3.9] bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-29230) (GH-29241) (GH-29245)
    f240714

    @miss-islington
    Copy link
    Contributor

    New changeset 14a4fce by m-aciek in branch 'main':
    bpo-45618: Update Sphinx version used to build the documentation to 4.2.0 (GH-29256)
    14a4fce

    @ned-deily
    Copy link
    Member

    New changeset 7a58509 by Ned Deily in branch '3.7':
    bpo-45618: Fix documentation build by pinning Docutils version to 0.17.1 (GH-31476)
    7a58509

    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Mar 24, 2022

    After the Jinja version 3.1.0 release today [1], documentation builds fail for all stable branches. Sphinx in versions pinned on those branches fails with ImportError. [2]

    Backporting 14a4fce to 3.10 and 3.9 fixes the problem for them (build is successful with 4.2.0). For 3.8 and 3.7 pinning down Jinja is required (docs don't build smoothly with 4.2.0 for those branches).

    Could some core developer help with the backports?

    I am starting to work on PRs with pins for 3.8 and 3.7.

    [1] https://jinja.palletsprojects.com/en/3.1.x/changes/#version-3-1-0
[2] % sphinx-build --version                                  
Traceback (most recent call last):
  File "…/cpython/Doc/./venv/bin/sphinx-build", line 5, in <module>
    from sphinx.cmd.build import main
  File "…/cpython/Doc/venv/lib/python3.9/site-packages/sphinx/cmd/build.py", line 25, in <module>
    from sphinx.application import Sphinx
  File "…/cpython/Doc/venv/lib/python3.9/site-packages/sphinx/application.py", line 42, in <module>
    from sphinx.registry import SphinxComponentRegistry
  File "…/cpython/Doc/venv/lib/python3.9/site-packages/sphinx/registry.py", line 24, in <module>
    from sphinx.builders import Builder
  File "…/cpython/Doc/venv/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 26, in <module>
    from sphinx.util import import_object, logging, rst, progress_message, status_iterator
  File "…/cpython/Doc/venv/lib/python3.9/site-packages/sphinx/util/rst.py", line 22, in <module>
    from jinja2 import environmentfilter
ImportError: cannot import name 'environmentfilter' from 'jinja2' (…/cpython/Doc/venv/lib/python3.9/site-packages/jinja2/__init__.py)

    @m-aciek m-aciek mannequin removed the 3.11 only security fixes label Mar 24, 2022
    @m-aciek m-aciek mannequin reopened this Mar 24, 2022
    @m-aciek m-aciek mannequin removed the 3.11 only security fixes label Mar 24, 2022
    @m-aciek m-aciek mannequin reopened this Mar 24, 2022
    @ned-deily
    Copy link
    Member

    This really should be handled in a new issue since the original fixes for all affected releases are already in the field. Moreover, we should be a bit careful about bumping the required Sphinx version for older releases as this has caused problems for downstream distributors of Python in the past. Can't we just pin the Jinja2 version for current releases?

    @m-aciek
    Copy link
    Mannequin Author

    m-aciek mannequin commented Mar 28, 2022

    This really should be handled in a new issue since the original fixes for all affected releases are already in the field.

    Sorry, I got suggested too much with expected fix which is common for both of those issues. Shall I create a new issue?

    Moreover, we should be a bit careful about bumping the required Sphinx version for older releases as this has caused problems for downstream distributors of Python in the past. Can't we just pin the Jinja2 version for current releases?

    Didn't know about the problems for distributors. Definitely just pinning Jinja2 is enough.

    @hugovk
    Copy link
    Member

    hugovk commented Mar 28, 2022

    > This really should be handled in a new issue since the original fixes for all affected releases are already in the field.

    Sorry, I got suggested too much with expected fix which is common for both of those issues. Shall I create a new issue?

    I created https://bugs.python.org/issue47138, let's continue there.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @m-aciek m-aciek mentioned this issue Aug 6, 2022
    Merged
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir release-blocker type-crash A hard crash of the interpreter, possibly with a core dump
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    5 participants
    @ambv @hugovk @ned-deily @pablogsal @miss-islington