classification
Title: Patch to enhance literal block language declaration
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Decorater, docs@python, georg.brandl, martin.panter, mdk, python-dev, r.david.murray, serhiy.storchaka, zach.ware
Priority: normal Keywords: patch

Created on 2016-02-29 23:58 by mdk, last changed 2016-08-05 08:47 by martin.panter. This issue is now closed.

Files
File name Uploaded Description Edit
literal_blocks_languages.patch mdk, 2016-02-29 23:58 Patch to enhance literal blocks languages
literal_blocks_languages.patch mdk, 2016-03-01 21:25
issue26462_regen.diff zach.ware, 2016-03-01 21:38 review
issue26462.diff mdk, 2016-07-23 22:24
issue26462.v3_regen.diff martin.panter, 2016-07-25 03:48 review
issue26462.diff mdk, 2016-07-25 08:02
issue26462.v4_regen.diff martin.panter, 2016-07-26 03:21 review
issue26462.v5.diff mdk, 2016-07-26 09:21
issue26462.v5_regen.diff martin.panter, 2016-07-28 07:39 review
issue26462.v6.diff mdk, 2016-07-28 09:27 review
issue26462.v7.diff mdk, 2016-07-28 13:48 review
Messages (30)
msg261035 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-02-29 23:58
Hi,

As I don't like warnings, and sphinx-doc was verbose about "Could not parse literal_block as "python3". highlighting skipped.", I fixed most of them.

Bonus: It's graphically better, as an example the XML block here: https://docs.python.org/3.5/library/pyexpat.html is now nicely colored: http://www.afpy.org/doc/python/3.5/library/pyexpat.html
msg261039 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2016-03-01 06:00
Thanks for the patch!

Most "bash" blocks should be "console" (since the prompt is shown). "pycon" should not be needed, since it is detected automatically.
msg261076 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-03-01 21:25
I revewed all my ".. code-block:: bash" and you're right (I was unaware of the existence of the "console" lexer). I changed them (reviewing them one by one) to ".. code-block:: shell-session", a synonym for ".. code-block:: console" (http://pygments.org/docs/lexers/?highlight=console#pygments.lexers.shell.BashSessionLexer) which I find, in a Python context, semantically more precise, and less ambigious, as one may read "console" as "the repl" (I assume most writers of the doc are aware of commonly used pygment lexers, it's still better for newcomers to be explicit and readable).

For your other point, about pycon being automatically detected, I tried removeing them and it works, I was sure I was adding them only where the python3 parser was reporting an error (was fixing errors one by one). I keeped the explicit pycon lexer in the "Doc/extending/extending.rst" file which have a ".. highlightlang:: c".

Here is the new patch, and thanks for this usefull review !
msg261079 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-03-01 21:38
Attaching regenerated patch to coax Rietveld into accepting it for easier review.
msg261097 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2016-03-02 06:21
Looks good now! I left a few comments on Rietveld.
msg271080 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-23 14:47
Hi,

Look like it has not been merged, is there something I can do to help with this?

Bests
msg271105 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-07-23 19:49
Hi Julien, there are still a few unanswered review comments.  Check the 'review' link on 'issue26462_regen.diff' (which was your patch, I just recreated it in a format that our Rietveld review tool would accept).
msg271117 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-23 22:24
Hi,

I completly missed the "review" link, sry. I reviewed comments, and I'm trying to provide an up-to-date patch that you'll hopefully won't have to doctor to make Rietveld eat it.
msg271232 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-25 05:05
With Julien’s patch applied, these are the remaining warnings:

/media/disk/home/proj/python/cpython/Doc/library/configparser.rst:240: WARNING: Could not lex literal_block as "ini". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/library/configparser.rst:301: WARNING: Could not lex literal_block as "ini". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/library/configparser.rst:332: WARNING: Could not lex literal_block as "ini". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/library/configparser.rst:341: WARNING: Could not lex literal_block as "ini". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/reference/lexical_analysis.rst:799: WARNING: Could not lex literal_block as "python3". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/reference/lexical_analysis.rst:813: WARNING: Could not lex literal_block as "python3". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/whatsnew/2.4.rst:1430: WARNING: Could not lex literal_block as "python3". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/whatsnew/3.5.rst:285: WARNING: Could not lex literal_block as "python3". Highlighting skipped.
/media/disk/home/proj/python/cpython/Doc/whatsnew/3.5.rst:294: WARNING: Could not lex literal_block as "python3". Highlighting skipped.

Also, perhaps you didn’t see Georg’s comment in some unpatched code: <http://bugs.python.org/review/26462/diff/16666/Doc/library/logging.config.rst#newcode703>
msg271241 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-25 08:02
Upladed a new patch, I was working on 3.5 branch so I did not get all errors.

I'm still having:

    WARNING: Could not parse literal_block as "ini". highlighting skipped.

But it may be considered another bug: pygments ini parser can't parse ini syntax used in configparser doc (probably due to the columns usage instead of equals signs): I prefer having an "can't parse ini" on an ini paragraph than "can't parse python3".
msg271242 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-25 08:04
Oh and, is there a proper way for me to generate a diff that rietveld can eat without someone regeneratig it?
msg271263 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-25 13:02
I think generating it with Mercurial (and not using Mercurial’s Git patch format) is the best way. I just left some more notes at <https://bugs.python.org/issue13963#msg271260>.
msg271332 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-26 06:36
I’ve finished going over the latest patch. There are a couple of problems; see the review link.

Also, why did you add the changes in Doc/library/decimal.rst? In your second patch, you removed one of these changes, but now you have added more. They seem to reduce the number of doctests reported (make -C Doc/ doctest).
msg271350 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-26 09:21
Here a new patch after reviewing comments on rietveld.
msg271351 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-26 09:26
@martin I reviewed my changes on `decimal.rst` and I now just fixing the indentation problem, so I don't change anything unrealated to fixing warnings, and don't break doctests.
msg271509 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-28 08:31
V5 looks pretty good to me. With your blessing of restoring the python -q example from v4 (see review), I think it is ready to commit.
msg271511 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-28 09:27
Hi,

You're right, nice catch!

Removing `python -q` from the code block demonstrating it was a bad idea. I fixed it in the v6.

Thanks!
msg271521 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-28 11:13
One last change I think needs making to the same demo, the “code-block” needs indenting under the bullet point:

* The interpreter can now be started with a quiet option, ``-q``, to prevent
  the copyright and version information from being displayed in the interactive
  mode.  The option can be introspected using the :attr:`sys.flags` attribute:
  
  .. code-block:: shell-session  <== Indented
     
    $ python -q

Otherwise, it seems to include extra text in the code block.
msg271548 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-28 13:48
Hi,

Indentation in whatsnew/3.2 fixed.

Colors in this block are clearly not perfect, but I think that's another subject.
msg271549 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-28 13:58
Hi,

Would you like me to also provide patches for different versions?
I only provided patches for the default branch, but I'll gladly see this applied on other branches, as I often build them all.

Also is a documented policy about maintaining the documentation?
I have some questions like:

- Should we maintain bugfix/security/end-of-life branches?
- Are we even allowed to push a documentation change to them?
- I'm seeing that the 3.x documentation are converging by using the "New in version 3.xx." marks, but they still different, is this effort documented? 
- Is this a goal to merge them on the long term?
- Is there a team on this?
msg271550 - (view) Author: Decorater (Decorater) * Date: 2016-07-28 14:09
tbh it would be nice if the entire documentation was recolored to look more 'interesting' to read. And to also have it in a way that people who lean visually can learn the info easier instead of them trying to read a giant wall of text that they may or may not understand. (got to hate reading giant walls) But yeah all versions on it should be good.
msg271551 - (view) Author: Decorater (Decorater) * Date: 2016-07-28 14:10
Also I was thinking maybe I could figure out how to Add in asyncio to 2.7 anyway (well latest one that is) because why not. With as many things using asyncio now days it would be cake for those python 2 users.
msg271553 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-28 14:23
Usually my technique is to apply the 3.6 patch to 3.5, fix up any conflicts, and leave the 3.6-only bits out (which get rejected by the patch process anyway). But dedicated patch(es) may be useful. Especially for 2.7, where there are probably independent changes to be made (e.g. modules that were removed in Python 3).

I think the policy on documentation in each branch should be in the devguide. My understanding: In general, bug fix branches (3.5, 2.7) have the documentation maintained, but generally not the older security-only branches.

My view is if a feature is added to (say) 3.6, then it gets a new-in-3.6 notice in the 3.6 documentation, but nothing gets added to 3.5. The Python 3 documentation rarely mentions features of Python 2, so every feature is treated as being new in 3.0 by default. For Python 2, it won’t document new features of Python 3, but is updated with changes relevant to porting to 3. So in theory the latest Python 3.6+ documentation should also be usable with 3.5, but not with Python 2. That’s about all I know :)
msg271554 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-07-28 14:32
Martin: your summary is correct.  A new feature should also have a What's New entry, of course; I think that's the only essential you left out.

Decorator: general comments (eg: color of docs, etc) aren't really useful comments on a specific bug report, and the asyncio comment is totally irrelevant.
msg271560 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-28 15:49
Martin: OK, let's apply this to 3.6 / 3.5 with this one, and I'll provide independent patch for 2.7 if needed (long time without building 2.7 doc).
msg271566 - (view) Author: Julien Palard (mdk) * (Python committer) Date: 2016-07-28 16:41
Decorater: Colors used by the documentation are defined here: https://github.com/sphinx-doc/sphinx/blob/master/sphinx/pygments_styles.py#L22 you can easily modify it and rebuild the doc ``(cd Doc; make html)``. Take a look at existing themes (https://help.farbox.com/pygments.html), sphinx uses a vaiarion of "friendly". But, in general, please open a new issue to speak on different subjects, typically about asyncio which is not related to this issue. Have a nice day.
msg271596 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-07-29 02:34
New changeset 8ba0000c90be by Martin Panter in branch '3.5':
Issue #26462: Doc: reduce literal_block warnings, fix syntax highlighting.
https://hg.python.org/cpython/rev/8ba0000c90be

New changeset 6fbb7c9d77c6 by Martin Panter in branch 'default':
Issue #26462: Merge code block fixes from 3.5
https://hg.python.org/cpython/rev/6fbb7c9d77c6
msg271597 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-07-29 03:28
Okay, leaving this open for a 2.7 patch. BTW thanks for your work; I tried to tackle these warnings a while ago, but I didn’t know enough about Sphix/RST/etc. The 2.7 change should be fairly easy; I can give it a crack if you’re not that interested in 2.7.
msg272004 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-08-05 02:16
New changeset 6e9a040a993f by Martin Panter in branch '2.7':
Issue #26462: Doc: avoid literal_block warnings, fix syntax highlighting.
https://hg.python.org/cpython/rev/6e9a040a993f
msg272019 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-08-05 08:47
I backported the patch to 2.7. Now I get no warnings about syntax highlighting from 2.7 (there were only a few reported before).
History
Date User Action Args
2016-08-05 08:47:32martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg272019

stage: needs patch -> resolved
2016-08-05 02:16:08python-devsetmessages: + msg272004
2016-07-29 03:28:28martin.pantersetversions: + Python 2.7
2016-07-29 03:28:09martin.pantersetmessages: + msg271597
stage: commit review -> needs patch
2016-07-29 02:34:42python-devsetnosy: + python-dev
messages: + msg271596
2016-07-28 16:41:22mdksetmessages: + msg271566
2016-07-28 15:49:58mdksetmessages: + msg271560
2016-07-28 14:32:31r.david.murraysetnosy: + r.david.murray
messages: + msg271554
2016-07-28 14:23:26martin.pantersetmessages: + msg271553
2016-07-28 14:10:37Decoratersetmessages: + msg271551
2016-07-28 14:09:08Decoratersetnosy: + Decorater
messages: + msg271550
2016-07-28 13:58:52mdksetmessages: + msg271549
2016-07-28 13:48:41mdksetfiles: + issue26462.v7.diff

messages: + msg271548
2016-07-28 11:13:27martin.pantersetmessages: + msg271521
2016-07-28 09:27:23mdksetfiles: + issue26462.v6.diff

messages: + msg271511
2016-07-28 08:31:45martin.pantersetmessages: + msg271509
stage: patch review -> commit review
2016-07-28 07:39:57martin.pantersetfiles: + issue26462.v5_regen.diff
2016-07-26 09:26:19mdksetmessages: + msg271351
2016-07-26 09:22:00mdksetfiles: + issue26462.v5.diff

messages: + msg271350
2016-07-26 06:36:06martin.pantersetmessages: + msg271332
2016-07-26 03:22:15martin.pantersetfiles: + issue26462.v4_regen.diff
2016-07-25 13:02:52martin.pantersetmessages: + msg271263
2016-07-25 08:04:14mdksetmessages: + msg271242
2016-07-25 08:02:40mdksetfiles: + issue26462.diff

messages: + msg271241
2016-07-25 05:05:42martin.pantersetnosy: + martin.panter
messages: + msg271232
2016-07-25 03:48:27martin.pantersetfiles: + issue26462.v3_regen.diff
2016-07-23 22:25:04mdksetfiles: + issue26462.diff

messages: + msg271117
2016-07-23 19:49:06zach.waresetmessages: + msg271105
2016-07-23 14:47:54mdksetmessages: + msg271080
2016-03-02 06:21:51georg.brandlsetmessages: + msg261097
2016-03-01 21:38:45zach.waresetfiles: + issue26462_regen.diff
nosy: + zach.ware
messages: + msg261079

2016-03-01 21:25:41mdksetfiles: + literal_blocks_languages.patch

messages: + msg261076
2016-03-01 06:00:34georg.brandlsetmessages: + msg261039
2016-03-01 05:37:24serhiy.storchakasetnosy: + georg.brandl, serhiy.storchaka
stage: patch review

versions: + Python 3.6
2016-02-29 23:58:50mdkcreate