classification
Title: docs: "yield from" breaks Pygments syntax coloring in doc examples
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.3, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Ramchandra Apte, amaury.forgeotdarc, berker.peksag, docs@python, georg.brandl, r.david.murray
Priority: normal Keywords:

Created on 2012-04-05 08:04 by Ramchandra Apte, last changed 2012-09-30 16:15 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
screenshot.png berker.peksag, 2012-04-05 13:07
Messages (10)
msg157551 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2012-04-05 08:04
In http://docs.python.org/dev/whatsnew/3.3.html#pep-380-syntax-for-delegating-to-a-subgenerator , two code samples
msg157552 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2012-04-05 08:04
Whoops - 
In http://docs.python.org/dev/whatsnew/3.3.html#pep-380-syntax-for-delegating-to-a-subgenerator , two code samples are not highlighted.
msg157579 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-05 12:59
I'm not seeing any unhighlighted examples in that section.
msg157584 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2012-04-05 13:07
I can reproduce. See screenshot.png.
msg157587 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-05 13:47
Berker: you can reproduce the bug, or the fact that they are highlighted?  The png looks like they are highlighted, so I assume the latter.
msg157589 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-04-05 13:56
> they are highlighted
No. Keywords normally appear in bold font, and with another color. This is not the case in these blocks.

Probably because the language detection does not work with the new "yield from" construct.  Sphinx should always use the latest Python...
msg157593 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-05 14:11
Ah, you mean they are not *syntax* highlighted.  Now I understand.  Sorry for missing that.

My understanding is that Sphinx does not use Python directly to parse the code and highlight it, it uses pygments, which uses regexes.  So this basically amounts to a feature request for pygments.
msg157595 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2012-04-05 14:32
Not exactly. Sphinx first tries to see if the block is a Python script, and to do so it uses the Python compiler:
https://bitbucket.org/birkenfeld/sphinx/src/164f59b2d946/sphinx/highlighting.py#cl-117
In case of SyntaxError, it treats the whole block as plain text and does not try to use pygmentq at all.
msg157596 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-04-05 14:44
Huh.  Well, in another issue Georg said it was now possible to upgrade the doc build toolchain to Python3.
msg171647 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-09-30 16:15
I've updated the version of Pygments used by 3.3 and 3.4 branches, which fixes this and the "raise from" issue.
History
Date User Action Args
2012-09-30 16:15:01georg.brandlsetstatus: open -> closed
versions: + Python 3.4, - Python 2.7, Python 3.2
nosy: + georg.brandl

messages: + msg171647

resolution: fixed
2012-09-29 17:40:13chris.jerdoneksettitle: docs: 2 code examples not Pygmented (syntax color coded) -> docs: "yield from" breaks Pygments syntax coloring in doc examples
2012-09-29 17:32:31chris.jerdoneklinkissue16084 superseder
2012-04-06 21:47:50terry.reedysettitle: docs:Code not highlighted -> docs: 2 code examples not Pygmented (syntax color coded)
stage: needs patch
type: behavior
versions: + Python 2.7, Python 3.2, Python 3.3
2012-04-05 14:44:58r.david.murraysetmessages: + msg157596
2012-04-05 14:32:18amaury.forgeotdarcsetmessages: + msg157595
2012-04-05 14:11:24r.david.murraysetmessages: + msg157593
2012-04-05 13:56:59amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg157589
2012-04-05 13:47:42r.david.murraysetmessages: + msg157587
2012-04-05 13:07:31berker.peksagsetfiles: + screenshot.png
nosy: + berker.peksag
messages: + msg157584

2012-04-05 12:59:13r.david.murraysetnosy: + r.david.murray
messages: + msg157579
2012-04-05 08:04:49Ramchandra Aptesetmessages: + msg157552
2012-04-05 08:04:13Ramchandra Aptecreate