classification
Title: Emit major version based canonical URLs for docs
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: Wheerd, berker.peksag, docs@python, ezio.melotti, georg.brandl, mbussonn, ncoghlan, python-dev
Priority: normal Keywords: patch

Created on 2016-02-13 08:52 by ncoghlan, last changed 2017-03-31 16:36 by dstufft. This issue is now closed.

Files
File name Uploaded Description Edit
cannonical-doc-for-3.4plus.patch mbussonn, 2017-02-07 18:04
cannonical-doc-for-2.7.patch mbussonn, 2017-02-07 18:21 review
cannonical-doc-for-2.6.patch mbussonn, 2017-02-07 18:36 review
Pull Requests
URL Status Linked Edit
PR 552 closed dstufft, 2017-03-31 16:36
Messages (22)
msg260227 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2016-02-13 08:52
Based on a recent comment on one of the mailing lists, I spent a bit of time looking into canonical URLs and their implications for how search engines treat versioned documentation.

The most relevant resource for that in relation to the CPython docs appears to be this page on ReadTheDocs: http://docs.readthedocs.org/en/latest/canonical.html

I sometimes encounter direct links to particular 3.x versions in search results, so I'm wondering if it might be desirable to set up the Python 3 docs to report their canonical URL as "/3/<path>", and the Python 2 docs as "/2/<path>".
msg260237 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2016-02-13 12:13
So we'd have to add a <link> tag? Should be easy.
msg286319 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-01-26 16:21
Belatedly following up on this, yeah, the RTFD page indicates that the header link should look like:

    <link rel="canonical" href="http://docs.python.org/3/<path>">
msg286336 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-01-26 22:26
Does this have to be implemented on the doc build of EOL pythons versions (like 2.6), or can it be a script which is ran once on these old docs ? 


One of the issues I had trying to implement that on other projects was that you don't know in advance what the future pages will be or if it will be gone. So you can go this route[1], in which case it's a 10 line fix that I'm happy to contribute also here. 

Typically `https://docs.python.org/2/c-api/string.html` has no cannonical on 3 whouch you can't know while building with sphinx. 

Or I though about injecting an html comment with a html comment that you search and replace once you publish the "new" version and only if the target page exists on the new version. 


So : 

- is that ok to have non existing cannonical ? 
- is that ok if adding <rel>s on docs is (or requires) a post-sphinx script ? 

1: https://github.com/xonsh/xonsh/pull/1914/files

Thanks
msg286339 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-01-27 00:14
Here is a patch that add a canonical link to the current documentation that should apply cleanly to 3.4 and above. I can do similar for older versions.
msg286342 - (view) Author: Manuel Krebber (Wheerd) * Date: 2017-01-27 08:09
I create the last diff without creating a commit, so maybe this one works better.
msg286344 - (view) Author: Manuel Krebber (Wheerd) * Date: 2017-01-27 08:12
Sorry, I accidentally replied to the worng issue -.-
msg287139 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-06 14:57
Thanks Matthias!

Regarding 2v3, the layout differences aren't a problem, since the canonical URLs are separate (/2/* vs /3/*). That's one of the benefits I actually hope for with this change - due to PEP 430, deep links still go to the Python 2 documentation by default, and once this change is made in the Python 2.7 branch it should teach search engines that those should start being presented in results as "/2/*" qualified links.

We also don't tend to make wholesale changes to the URL layouts in the docs in X.Y releases, so I think the assumption of "the relative path of this page won't change" is fine.

Georg, Berker - any further thoughts before we make this change to 3.4+ and the 2.7 docs?
msg287191 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-02-07 04:59
Thanks Nick, 

I can work on the similar patch for Python 2.7.
I'll wait for the migration to GitHub which IIRC should be soon.

Side question, is there some metrics (like google analytics) to know how much traffic there is on older Python docs and if it's worth patching ?
msg287203 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-02-07 07:36
Patch looks good to me too, thanks! Just left a super minor comment on Rietveld.

You might want to commit this to 3.5+ because we don't daily build 3.4 docs anymore: https://github.com/python/docsbuild-scripts/blob/master/build_docs.py#L30

I didn't see any links from 2.6 or older Python 2 versions in search results before so I don't have a strong opinion on backporting the patch to 2.7 (I probably wouldn't bother with Python 2 docs anymore :))
msg287226 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-07 11:00
OK, I've marked 2.7 and 3.5+ as the minimal set of versions to get the change via the CPython source repo.

However, looking at the results of https://www.google.com/search?q=python+httplib and https://www.google.com/search?q=python+http+client I think it's going to be worth backfilling the old branches as well - having multiple different versions of the 3.x documentation showing up in search results isn't helpful to anyone.
msg287249 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-02-07 18:04
Updated patch to take comments into account (added space before /> for consistency). I'm still unfamiliar with hg so let me know if I did anything wrong.
msg287419 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-09 14:23
Looks good to me, so I'll apply these right now :)
msg287425 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-09 15:09
New changeset c63b09833141 by Nick Coghlan in branch '3.5':
Issue #26355: Specify canonical URLs in docs pages
https://hg.python.org/cpython/rev/c63b09833141

New changeset 80970cf56048 by Nick Coghlan in branch '3.6':
Merge issue #26355 fix from Python 3.5
https://hg.python.org/cpython/rev/80970cf56048

New changeset 26af402c291f by Nick Coghlan in branch 'default':
Merge issue #26355 fix from 3.6
https://hg.python.org/cpython/rev/26af402c291f
msg287430 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-09 16:00
New changeset a47e20b636d2a5559e5831c6805df3cba1ddb2a1 by Nick Coghlan in branch 'master':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/a47e20b636d2a5559e5831c6805df3cba1ddb2a1

New changeset 37150972faf660571a3ae7076a623087c06b8791 by Nick Coghlan in branch 'master':
Merge issue #26355 fix from Python 3.5
https://github.com/python/cpython/commit/37150972faf660571a3ae7076a623087c06b8791

New changeset d267bc695eaf9422668daedb9e44442696e01fe7 by Nick Coghlan in branch 'master':
Merge issue #26355 fix from 3.6
https://github.com/python/cpython/commit/d267bc695eaf9422668daedb9e44442696e01fe7
msg287431 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-09 16:00
New changeset a47e20b636d2a5559e5831c6805df3cba1ddb2a1 by Nick Coghlan in branch '3.5':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/a47e20b636d2a5559e5831c6805df3cba1ddb2a1
msg287432 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-09 16:00
New changeset b07d454e45a2 by Nick Coghlan in branch '2.7':
Issue #26355: Specify canonical URLs in docs pages
https://hg.python.org/cpython/rev/b07d454e45a2
msg287433 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-09 16:07
OK, I'm marking this as closed, since it's as resolved as we can make it through a *CPython* change.

Since the old branches aren't autobuilt anymore, adding a canonical URL reference to them would presumably be a matter of running a script over the built docs.
msg287437 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2017-02-09 17:00
New changeset e8455e7137b9bd63d4c0183558161dba6ea00d32 by Nick Coghlan in branch '2.7':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/e8455e7137b9bd63d4c0183558161dba6ea00d32
msg287441 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-02-09 17:55
> OK, I'm marking this as closed, since it's as resolved as we can make it through a *CPython* change.

Thanks you Nick, I appreciate the time you took to do that. ANd thank you Berker for the review.
msg287445 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-02-09 18:04
Matthias - thanks for figuring out how to turn my "we should do this" idea into a change we've actually made :)
msg287448 - (view) Author: Matthias Bussonnier (mbussonn) * Date: 2017-02-09 18:07
> turn my "we should do this" idea into a change we've actually made :)

Looking forward to being able to do this more on GitHub as I am more familiar with git.

Good luck for the transition, I'm pretty sure there will be some hard time in the next few weeks.
History
Date User Action Args
2017-03-31 16:36:30dstufftsetpull_requests: + pull_request1028
2017-02-09 18:07:45mbussonnsetmessages: + msg287448
2017-02-09 18:04:21ncoghlansetmessages: + msg287445
2017-02-09 17:55:09mbussonnsetmessages: + msg287441
2017-02-09 17:00:23python-devsetmessages: + msg287437
2017-02-09 16:07:27ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg287433

stage: commit review -> resolved
2017-02-09 16:00:51python-devsetmessages: + msg287432
2017-02-09 16:00:37python-devsetmessages: + msg287431
2017-02-09 16:00:34python-devsetmessages: + msg287430
2017-02-09 15:09:19python-devsetnosy: + python-dev
messages: + msg287425
2017-02-09 14:23:47ncoghlansetmessages: + msg287419
2017-02-09 14:22:31ncoghlansetassignee: docs@python -> ncoghlan
stage: patch review -> commit review
2017-02-07 18:36:53mbussonnsetfiles: + cannonical-doc-for-2.6.patch
2017-02-07 18:21:29mbussonnsetfiles: + cannonical-doc-for-2.7.patch
2017-02-07 18:04:39mbussonnsetfiles: + cannonical-doc-for-3.4plus.patch

messages: + msg287249
2017-02-07 18:02:52mbussonnsetfiles: - cannonical-doc-for-3.4plus.patch
2017-02-07 11:00:15ncoghlansetmessages: + msg287226
versions: + Python 2.7, Python 3.5, Python 3.6, Python 3.7
2017-02-07 07:36:10berker.peksagsetmessages: + msg287203
2017-02-07 04:59:50mbussonnsetmessages: + msg287191
2017-02-06 14:57:03ncoghlansetmessages: + msg287139
2017-02-06 07:34:55berker.peksagsetnosy: + berker.peksag

stage: needs patch -> patch review
2017-01-27 08:12:23Wheerdsetmessages: + msg286344
2017-01-27 08:10:11Wheerdsetfiles: - slot-wrapper-types.patch
2017-01-27 08:09:41Wheerdsetfiles: + slot-wrapper-types.patch
nosy: + Wheerd
messages: + msg286342

2017-01-27 00:14:35mbussonnsetfiles: + cannonical-doc-for-3.4plus.patch
keywords: + patch
messages: + msg286339
2017-01-26 22:26:46mbussonnsetnosy: + mbussonn
messages: + msg286336
2017-01-26 16:21:37ncoghlansetmessages: + msg286319
2016-02-13 12:13:49georg.brandlsetmessages: + msg260237
2016-02-13 10:24:40ezio.melottisetnosy: + ezio.melotti
2016-02-13 08:52:32ncoghlansetassignee: docs@python

components: + Documentation
nosy: + docs@python
2016-02-13 08:52:19ncoghlancreate