This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: disable test_robotparser test that uses an invalid URL
Type: behavior Stage: resolved
Components: Versions: Python 3.4, Python 3.5
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: orsenthil Nosy List: larry, orsenthil, python-dev, r.david.murray, vajrasky
Priority: high Keywords: patch

Created on 2014-02-23 23:25 by larry, last changed 2022-04-11 14:57 by admin. This issue is now closed.

File name Uploaded Description Edit
test_robotparser_use_local_server.patch vajrasky, 2014-02-24 09:50 review
Messages (6)
msg212044 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2014-02-23 23:25
Lib/tests/ uses the following url:

That file now serves a 404.

This should be disabled for 3.4, and the test rewritten for 3.5.  If we get a patch quickly I would probably cherry-pick it for 3.4 final.

Aside: it is REALLY IRRITATING that we have tests in the standard library that hit external web sites.  The standard library regression test suite should *not* rely on network access.  Can the rewritten test please spin up a local web server (http.server) and serve a robotx.txt that way?
msg212047 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-02-24 00:02
There are tests where we have good reasons why we hit external servers, and this is why the 'network' resource is not on by default.

That said, this doesn't appear to be a test that has a good reason to hit an external resource, since it should certainly be possible to serve a password protected 'site' locally.
msg212082 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-02-24 09:50
Okay, here is my attempt to rewrite the test using local web server. I copied and modified the code from Lib/test/ The website mueblesmoraleda can not be accessed so I could only guess what the intention of this test was.

I am not sure whether I should remove the duplicate comment and using daemon or not.
msg221538 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-06-25 09:59
New changeset 16d8240ff841 by Senthil Kumaran in branch '3.4':
issue20753 - robotparser tests should not rely upon external resource when not required.

New changeset 74cd8abcc302 by Senthil Kumaran in branch 'default':
merge from 3.4
msg221539 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2014-06-25 10:00
The patch was good and captured what was required. I made some minor modifications and committed it in 3.4 and 3.5

Thanks for the patch, Vajrasky Kok.
msg221838 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-06-29 12:56
New changeset 0e08ca451b34 by Berker Peksag in branch '3.4':
Issue #20753: Skip PasswordProtectedSiteTestCase when Python is built without threads.

New changeset 394e6bda5a70 by Berker Peksag in branch 'default':
Issue #20753: Merge with 3.4.
Date User Action Args
2022-04-11 14:57:59adminsetgithub: 64952
2014-06-29 12:56:21python-devsetmessages: + msg221838
2014-06-25 10:00:15orsenthilsetstatus: open -> closed

assignee: orsenthil

nosy: + orsenthil
messages: + msg221539
resolution: fixed
stage: needs patch -> resolved
2014-06-25 09:59:03python-devsetnosy: + python-dev
messages: + msg221538
2014-02-24 09:50:29vajraskysetfiles: + test_robotparser_use_local_server.patch

nosy: + vajrasky
messages: + msg212082

keywords: + patch
2014-02-24 00:02:58r.david.murraysetnosy: + r.david.murray
messages: + msg212047
2014-02-23 23:25:33larrysettitle: disable test_robotparser -> disable test_robotparser test that uses an invalid URL
2014-02-23 23:25:13larrycreate