`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives. #67252

msabramo · 2014-12-16T07:58:05Z

BPO	23063
Nosy	@merwok, @berkerpeksag, @msabramo, @dstufft, @posita
Files	cfabe07bc98f.diff 9b8f6812ff69.diff issue23063.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-01-15.05:00:39.460>
created_at = <Date 2014-12-16.07:58:05.153>
labels = ['type-bug', 'library']
title = '`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives.'
updated_at = <Date 2016-12-15.16:08:02.354>
user = 'https://github.com/msabramo'

bugs.python.org fields:

activity = <Date 2016-12-15.16:08:02.354>
actor = 'posita'
assignee = 'none'
closed = True
closed_date = <Date 2015-01-15.05:00:39.460>
closer = 'python-dev'
components = ['Distutils']
creation = <Date 2014-12-16.07:58:05.153>
creator = 'Marc.Abramowitz'
dependencies = []
files = ['37464', '37471', '37472']
hgrepos = ['288']
issue_num = 23063
keywords = ['patch']
message_count = 9.0
messages = ['232720', '232721', '232722', '232734', '232753', '232754', '232756', '232943', '234051']
nosy_count = 6.0
nosy_names = ['eric.araujo', 'python-dev', 'berker.peksag', 'Marc.Abramowitz', 'dstufft', 'posita']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23063'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

msabramo · 2014-12-16T07:58:04Z

python setup.py check --restructuredtext --strict --metadata fails with:

warning: check: Could not finish the parsing.

if the RST document uses code or code-block directives.

This is annoying because the document is valid, but it appears to be invalid
and confuses people. For example, see
ionelmc/pytest-benchmark#4 (comment)

How to reproduce this bug
-------------------------

Clone a repo that has a README.rst with code-block directives in it. E.g.:

    $ git clone git@github.com:ionelmc/pytest-benchmark.git
    $ cd pytest-benchmark
    $ git checkout ab0b08f6fccb06a7909905a8409f8faa8b01e0d8

Observe that it has "code-blocks" in it:

       $ grep 'code-block' README.rst
       .. code-block:: python
       .. code-block:: python

Observe that RST document is valid, according to rst2html.py:

       $ rst2html.py --halt=1 README.rst > README.html && echo "RST was OK."
       RST was OK.

       $ head -n 3 README.html
       <?xml version="1.0" encoding="utf-8" ?>
       <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
       <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

Observe that python setup.py check --restructuredtext --strict --metadata fails:

    $ python setup.py check --restructuredtext --strict --metadata
    running check
    warning: check: Could not finish the parsing.

error: Please correct your package.

    $ echo $?
    1

What was expected: python setup.py check --restructuredtext --strict --metadata should succeed with no warnings, just as rst2html.py did, because
README.rst is a valid RST document.

What actually happened: python setup.py check --restructuredtext --strict --metadata prints a warning and an error and fails, unlike rst2html.py

The error is coming from here:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L142

It's happening because of this line:
https://github.com/python/cpython/blob/master/Lib/distutils/command/check.py#L125

:::python
settings = frontend.OptionParser().get_default_values()

If this is changed to:

:::python
settings = frontend.OptionParser(components=(Parser,)).get_default_values()

then things work much better (this is how tools/quicktest.py in docutils does it for example -- see https://sourceforge.net/p/docutils/code/HEAD/tree/trunk/docutils/tools/quicktest.py#l196)

The attached patch prevents the failure from happening and also adds more information to the error when things go south.

msabramo · 2014-12-16T08:10:00Z

Testing a few common cases:

$ python setup.py check --restructuredtext --strict --metadata
running check
error: The docutils package is needed.

$ python setup.py check --restructuredtext --strict --metadata
running check
warning: check: Cannot analyze code. Pygments package not found. (line 66)

warning: check: Cannot analyze code. Pygments package not found. (line 99)

error: Please correct your package.

$ python setup.py check --restructuredtext --strict --metadata && echo "RST was OK."
running check
RST was OK.

msabramo · 2014-12-16T08:14:06Z

Note that this patch does 2 things:

Improves the error message
Prevents check from failing when there are code-blocks

If I only did #1 and not #2, then output looks like this:

$ python setup.py check --restructuredtext --strict --metadata && echo "RST was OK."
running check
warning: check: Could not finish the parsing: 'Values' object has no attribute 'syntax_highlight'.

error: Please correct your package.

berkerpeksag · 2014-12-16T11:55:50Z

Could you add a test? See Lib/distutils/tests/test_check.py.

msabramo · 2014-12-16T18:14:49Z

OK, I added a test.

See:

If you want, I can also update the diff attached here, but maybe it's easier to just look at my branch on Bitbucket? Whatever is most convenient for you...

merwok · 2014-12-16T18:18:49Z

The Mercurial integration lets you update the patch with one click. Decentralized VCSes are nice :‑)

Patch looks good to me. Thank you.

msabramo · 2014-12-16T18:41:21Z

What's the next step?

merwok · 2014-12-19T17:34:27Z

An active core developer needs to see this and decide to commit the patch. You can see if there’s someone on IRC. If there’s no action in a week or two, feel free to ask on python-dev (maybe listing more than one waiting patch, or offering reviews in exchange for core dev time).

python-dev · 2015-01-15T05:00:39Z

New changeset db09d760b965 by Benjamin Peterson in branch '3.4':
fix parsing reST with code or code-block directives (closes bpo-23063)
https://hg.python.org/cpython/rev/db09d760b965

New changeset 38826e21f0db by Benjamin Peterson in branch '2.7':
fix parsing reST with code or code-block directives (closes bpo-23063)
https://hg.python.org/cpython/rev/38826e21f0db

New changeset 731a36c13629 by Benjamin Peterson in branch 'default':
merge 3.4 (bpo-23063)
https://hg.python.org/cpython/rev/731a36c13629

msabramo mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Dec 16, 2014

python-dev mannequin closed this as completed Jan 15, 2015

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives. #67252

`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives. #67252

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

berkerpeksag commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

merwok commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

merwok commented Dec 19, 2014

python-dev mannequin commented Jan 15, 2015

python setup.py check --restructuredtext --strict --metadata fails with: warning: check: Could not finish the parsing. if the RST document uses code or code-block directives. #67252

python setup.py check --restructuredtext --strict --metadata fails with: warning: check: Could not finish the parsing. if the RST document uses code or code-block directives. #67252

Comments

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

berkerpeksag commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

merwok commented Dec 16, 2014

msabramo mannequin commented Dec 16, 2014

merwok commented Dec 19, 2014

python-dev mannequin commented Jan 15, 2015

`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives. #67252

`python setup.py check --restructuredtext --strict --metadata` fails with: `warning: check: Could not finish the parsing.` if the RST document uses code or code-block directives. #67252