Message152446
http://hg.python.org/robots.txt currently disallows all robots from all paths. This means that the site doesn't show up in Google search results seeking, for instance, browsing access to the python source
https://www.google.com/search?ie=UTF-8&q=python+source+browse
https://www.google.com/search?ie=UTF-8&q=python+repo+browse
https://www.google.com/search?ie=UTF-8&q=hg+python+browse
etc...
Instead, robots.txt should allow access to the landing page, http://hg.python.org/, and the landing pages for hosted projects, e.g. http://hg.python.org/cpython/, while prohibiting access to the */rev/*, */shortlog/*, ..., directories.
This change would be very easy, cost virtually nothing, and let users find the mercurial repository viewer from search engines. Note that http://svn.python.org/ does show up in search results, as an illustration of how convenient this is. |
|
Date |
User |
Action |
Args |
2012-02-01 22:29:56 | Ivaylo.Popov | set | recipients:
+ Ivaylo.Popov |
2012-02-01 22:29:56 | Ivaylo.Popov | set | messageid: <1328135396.1.0.0965106732225.issue13924@psf.upfronthosting.co.za> |
2012-02-01 22:29:55 | Ivaylo.Popov | link | issue13924 messages |
2012-02-01 22:29:54 | Ivaylo.Popov | create | |
|