Based on a recent comment on one of the mailing lists, I spent a bit of time looking into canonical URLs and their implications for how search engines treat versioned documentation.

The most relevant resource for that in relation to the CPython docs appears to be this page on ReadTheDocs: http://docs.readthedocs.org/en/latest/canonical.html

I sometimes encounter direct links to particular 3.x versions in search results, so I'm wondering if it might be desirable to set up the Python 3 docs to report their canonical URL as "/3/<path>", and the Python 2 docs as "/2/<path>".

So we'd have to add a <link> tag? Should be easy.

Belatedly following up on this, yeah, the RTFD page indicates that the header link should look like:

    <link rel="canonical" href="http://docs.python.org/3/<path>">

Does this have to be implemented on the doc build of EOL pythons versions (like 2.6), or can it be a script which is ran once on these old docs ? 


One of the issues I had trying to implement that on other projects was that you don't know in advance what the future pages will be or if it will be gone. So you can go this route[1], in which case it's a 10 line fix that I'm happy to contribute also here. 

Typically `https://docs.python.org/2/c-api/string.html` has no cannonical on 3 whouch you can't know while building with sphinx. 

Or I though about injecting an html comment with a html comment that you search and replace once you publish the "new" version and only if the target page exists on the new version. 


So : 

- is that ok to have non existing cannonical ? 
- is that ok if adding <rel>s on docs is (or requires) a post-sphinx script ? 

1: https://github.com/xonsh/xonsh/pull/1914/files

Thanks

Here is a patch that add a canonical link to the current documentation that should apply cleanly to 3.4 and above. I can do similar for older versions.

I create the last diff without creating a commit, so maybe this one works better.

Sorry, I accidentally replied to the worng issue -.-

Thanks Matthias!

Regarding 2v3, the layout differences aren't a problem, since the canonical URLs are separate (/2/* vs /3/*). That's one of the benefits I actually hope for with this change - due to PEP 430, deep links still go to the Python 2 documentation by default, and once this change is made in the Python 2.7 branch it should teach search engines that those should start being presented in results as "/2/*" qualified links.

We also don't tend to make wholesale changes to the URL layouts in the docs in X.Y releases, so I think the assumption of "the relative path of this page won't change" is fine.

Georg, Berker - any further thoughts before we make this change to 3.4+ and the 2.7 docs?

Thanks Nick, 

I can work on the similar patch for Python 2.7.
I'll wait for the migration to GitHub which IIRC should be soon.

Side question, is there some metrics (like google analytics) to know how much traffic there is on older Python docs and if it's worth patching ?

Patch looks good to me too, thanks! Just left a super minor comment on Rietveld.

You might want to commit this to 3.5+ because we don't daily build 3.4 docs anymore: https://github.com/python/docsbuild-scripts/blob/master/build_docs.py#L30

I didn't see any links from 2.6 or older Python 2 versions in search results before so I don't have a strong opinion on backporting the patch to 2.7 (I probably wouldn't bother with Python 2 docs anymore :))

OK, I've marked 2.7 and 3.5+ as the minimal set of versions to get the change via the CPython source repo.

However, looking at the results of https://www.google.com/search?q=python+httplib and https://www.google.com/search?q=python+http+client I think it's going to be worth backfilling the old branches as well - having multiple different versions of the 3.x documentation showing up in search results isn't helpful to anyone.

Updated patch to take comments into account (added space before /> for consistency). I'm still unfamiliar with hg so let me know if I did anything wrong.

Looks good to me, so I'll apply these right now :)

New changeset c63b09833141 by Nick Coghlan in branch '3.5':
Issue #26355: Specify canonical URLs in docs pages
https://hg.python.org/cpython/rev/c63b09833141

New changeset 80970cf56048 by Nick Coghlan in branch '3.6':
Merge issue #26355 fix from Python 3.5
https://hg.python.org/cpython/rev/80970cf56048

New changeset 26af402c291f by Nick Coghlan in branch 'default':
Merge issue #26355 fix from 3.6
https://hg.python.org/cpython/rev/26af402c291f

New changeset a47e20b636d2a5559e5831c6805df3cba1ddb2a1 by Nick Coghlan in branch 'master':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/a47e20b636d2a5559e5831c6805df3cba1ddb2a1

New changeset 37150972faf660571a3ae7076a623087c06b8791 by Nick Coghlan in branch 'master':
Merge issue #26355 fix from Python 3.5
https://github.com/python/cpython/commit/37150972faf660571a3ae7076a623087c06b8791

New changeset d267bc695eaf9422668daedb9e44442696e01fe7 by Nick Coghlan in branch 'master':
Merge issue #26355 fix from 3.6
https://github.com/python/cpython/commit/d267bc695eaf9422668daedb9e44442696e01fe7

New changeset a47e20b636d2a5559e5831c6805df3cba1ddb2a1 by Nick Coghlan in branch '3.5':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/a47e20b636d2a5559e5831c6805df3cba1ddb2a1

New changeset b07d454e45a2 by Nick Coghlan in branch '2.7':
Issue #26355: Specify canonical URLs in docs pages
https://hg.python.org/cpython/rev/b07d454e45a2

OK, I'm marking this as closed, since it's as resolved as we can make it through a *CPython* change.

Since the old branches aren't autobuilt anymore, adding a canonical URL reference to them would presumably be a matter of running a script over the built docs.

New changeset e8455e7137b9bd63d4c0183558161dba6ea00d32 by Nick Coghlan in branch '2.7':
Issue #26355: Specify canonical URLs in docs pages
https://github.com/python/cpython/commit/e8455e7137b9bd63d4c0183558161dba6ea00d32

> OK, I'm marking this as closed, since it's as resolved as we can make it through a *CPython* change.

Thanks you Nick, I appreciate the time you took to do that. ANd thank you Berker for the review.

Matthias - thanks for figuring out how to turn my "we should do this" idea into a change we've actually made :)

> turn my "we should do this" idea into a change we've actually made :)

Looking forward to being able to do this more on GitHub as I am more familiar with git.

Good luck for the transition, I'm pretty sure there will be some hard time in the next few weeks.