➜

This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author calvin
Recipients
Date 2003-09-29.13:24:08
SpamBayes Score
Marked as misclassified
Message-id
In-reply-to
Content 
Logged In: YES 
user_id=9205

http://www.robotstxt.org/wc/norobots-rfc.html specifies the
401 and 403 return code consequences as restricting the
whole site (ie disallow_all = 1).

For the password input, the patch looks good to me. On the
long term, robotparser.py should switch to urllib2.py
anyway, and it should handle Transfer-Encoding: gzip.
History
Date User Action Args
2007-08-23 14:17:15adminlinkissue813986 messages
2007-08-23 14:17:15admincreate