Author karlcow
Recipients BreamoreBoy, dualbus, ezio.melotti, karlcow, orsenthil, terry.reedy, tshepang
Date 2014-06-22.21:52:01
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1403473921.64.0.382796379574.issue15851@psf.upfronthosting.co.za>
In-reply-to
Content
Mark,

The code is using urllib for demonstrating the issue with wikipedia and other sites which are blocking python-urllib user agents because it is used by many spam harvesters.

The proposal is about giving a possibility in robotparser lib to add a feature for setting up the user-agent.
History
Date User Action Args
2014-06-22 21:52:01karlcowsetrecipients: + karlcow, terry.reedy, orsenthil, ezio.melotti, BreamoreBoy, tshepang, dualbus
2014-06-22 21:52:01karlcowsetmessageid: <1403473921.64.0.382796379574.issue15851@psf.upfronthosting.co.za>
2014-06-22 21:52:01karlcowlinkissue15851 messages
2014-06-22 21:52:01karlcowcreate