This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author Ivaylo.Popov
Recipients Ivaylo.Popov
Date 2012-02-01.22:29:54
SpamBayes Score 0.00070951803
Marked as misclassified No
Message-id <1328135396.1.0.0965106732225.issue13924@psf.upfronthosting.co.za>
In-reply-to
Content
http://hg.python.org/robots.txt currently disallows all robots from all paths. This means that the site doesn't show up in Google search results seeking, for instance, browsing access to the python source
https://www.google.com/search?ie=UTF-8&q=python+source+browse
https://www.google.com/search?ie=UTF-8&q=python+repo+browse
https://www.google.com/search?ie=UTF-8&q=hg+python+browse
etc...

Instead, robots.txt should allow access to the landing page, http://hg.python.org/, and the landing pages for hosted projects, e.g. http://hg.python.org/cpython/, while prohibiting access to the */rev/*, */shortlog/*, ..., directories.

This change would be very easy, cost virtually nothing, and let users find the mercurial repository viewer from search engines. Note that http://svn.python.org/ does show up in search results, as an illustration of how convenient this is.
History
Date User Action Args
2012-02-01 22:29:56Ivaylo.Popovsetrecipients: + Ivaylo.Popov
2012-02-01 22:29:56Ivaylo.Popovsetmessageid: <1328135396.1.0.0965106732225.issue13924@psf.upfronthosting.co.za>
2012-02-01 22:29:55Ivaylo.Popovlinkissue13924 messages
2012-02-01 22:29:54Ivaylo.Popovcreate