This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_urllib2_localnet.test_bad_address fails: OSError not raised by urlopen
Type: Stage: resolved
Components: Tests Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Steve Harris, berker.peksag, martin.panter, python-dev, r.david.murray, raniere, vstinner
Priority: normal Keywords: patch

Created on 2015-03-27 12:18 by raniere, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
bad_address.patch martin.panter, 2015-12-08 07:56 review
Messages (14)
msg239390 - (view) Author: Raniere Silva (raniere) Date: 2015-03-27 12:18
Starting from Git commit 1996f3038458df2b6443c23e4705478ff5db43f8

$ ./configure
$ make
$ make test
======================================================================
FAIL: test_bad_address (test.test_urllib2_localnet.TestUrlopen)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/raniere/src/cpython/Lib/test/test_urllib2_localnet.py", line 656, in test_bad_address
    "http://sadflkjsasf.i.nvali.d./")
AssertionError: OSError not raised by urlopen

----------------------------------------------------------------------
msg239391 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-27 12:41
> Starting from Git commit 1996f3038458df2b6443c23e4705478ff5db43f8

https://github.com/python/cpython/commit/1996f3038458df2b6443c23e4705478ff5db43f8

=> https://hg.python.org/cpython/rev/b7c0137cccbe
msg239392 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-27 12:43
Hi,

Can you please complete your bug report? Can you specifiy your OS? Can you try to rerun the test manually with "./python -m test -u all test_urllib2_localnet"?

The test:

    def test_bad_address(self):
        # Make sure proper exception is raised when connecting to a bogus
        # address.

        # as indicated by the comment below, this might fail with some ISP,
        # so we run the test only when -unetwork/-uall is specified to
        # mitigate the problem a bit (see #17564)
        support.requires('network')
        self.assertRaises(OSError,
                          # Given that both VeriSign and various ISPs have in
                          # the past or are presently hijacking various invalid
                          # domain name requests in an attempt to boost traffic
                          # to their own sites, finding a domain name to use
                          # for this test is difficult.  RFC2606 leads one to
                          # believe that '.invalid' should work, but experience
                          # seemed to indicate otherwise.  Single character
                          # TLDs are likely to remain invalid, so this seems to
                          # be the best choice. The trailing '.' prevents a
                          # related problem: The normal DNS resolver appends
                          # the domain names from the search path if there is
                          # no '.' the end and, and if one of those domains
                          # implements a '*' rule a result is returned.
                          # However, none of this will prevent the test from
                          # failing if the ISP hijacks all invalid domain
                          # requests.  The real solution would be to be able to
                          # parameterize the framework with a mock resolver.
                          urllib.request.urlopen,
                          "http://sadflkjsasf.i.nvali.d./")
msg239398 - (view) Author: Raniere Silva (raniere) Date: 2015-03-27 13:07
> Can you please complete your bug report? Can you specifiy your OS?

GNU/Linux.

$ uname -a
Linux pupunha 3.19.2-1-ARCH #1 SMP PREEMPT Wed Mar 18 16:21:02 CET 2015 x86_64 GNU/Linux
$ gcc --version
gcc (GCC) 4.9.2 20150304 (prerelease)

> Can you try to rerun the test manually with "./python -m test -u all test_urllib2_localnet"?

$ ./python -m test -u all test_urllib2_localnet
[1/1] test_urllib2_localnet
test test_urllib2_localnet failed -- Traceback (most recent call last):
  File "/home/raniere/src/cpython/Lib/test/test_urllib2_localnet.py", line 656, in test_bad_address
    "http://sadflkjsasf.i.nvali.d./")
AssertionError: OSError not raised by urlopen

1 test failed:
    test_urllib2_localnet
msg239409 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-03-27 15:01
How did you determine it was that commit?  It's a doc update, so it can't be affecting the code.  The test passes for me, on linux.
msg239411 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-03-27 15:12
Can you try simply:

   socket.create_connection(("sadflkjsasf.i.nvali.d.", 80))

Here is raise: socket.gaierror: [Errno -2] Name or service not known

And what are your DNS servers?

   cat /etc/resolv.conf
msg254945 - (view) Author: Steve Harris (Steve Harris) Date: 2015-11-20 00:41
I experienced the same problem on my system:

OS: Xubuntu/Linux 12.04
$ uname -a
Linux ives 3.2.0-94-generic #134-Ubuntu SMP Fri Nov 6 18:17:08 UTC 2015 i686 i686 i386 GNU/Linux
$ gcc --version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3

=================================================

In response to your last request, here's what I get:

$ ./python 
Python 3.5.0 (default, Nov 19 2015, 17:12:32) 
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.create_connection(("sadflkjsasf.i.nvali.d.", 80))
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.8', 54128), raddr=('92.242.140.21', 80)>
>>> 

=================================================

And:

$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search domain.actdsltmp

=================================================

My ISP is Verizon.

Hope this is help
msg256051 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-07 06:25
Steve and/or Raniere: I suspect this is the same problem as mentioned in the giant comment, Issue 17564. In other words, it is probably your ISP or DNS which is broken, not Python.

However, rather than expecting everyone to automatically read the giant wall of comment in the source code, perhaps we could improve the failure message:

with self.assertRaises(OSError,
        msg="Perhaps your ISP hijacked the DNS request; "
        "see the source code comments"):
    # Giant comment
    urllib.request.urlopen(...)

Then the failure would look like this:

AssertionError: OSError not raised : Perhaps your ISP hijacked the DNS request; see the source code comments
msg256052 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-07 06:58
Actually test_bad_address() in /test/test_urllib2_localnet.py looks redundant with the test of the same name in /test/test_urllibnet.py. That test already has a nice precondition to skip the test if DNS does not fail, originating in revision 7d69d04522e3. And it has a failure message like I already proposed (added by Issue 3583, but probably not so important because that report was made before precondition in revision 7d69d04522e3 was added).

Long story short: Delete the redundant test from test_urllib2_localnet.

Makes me wonder what other redundancies there are. I guess this is leftover baggage from Python 2’s dual urllib and urllib2 modules.
msg256057 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-07 11:09
Sorry the revision adding the precondition is 6adab7448272.
msg256102 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-08 07:56
I propose this patch which merges test_bad_address() from test_urllib2_localnet.py into test_urllibnet.py. It includes applying r75111 to test_urllibnet.py, which adds a trailing dot to the domain name.
msg256506 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-12-16 03:25
bad_address.patch looks good to me.
msg256510 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-12-16 05:27
New changeset e81189f75d04 by Martin Panter in branch '3.5':
Issue #23788: Merge redundant test_bad_address() into test_urllibnet
https://hg.python.org/cpython/rev/e81189f75d04

New changeset 16accac4b2f6 by Martin Panter in branch 'default':
Issue #23788: Merge test_bad_address() cleanup from 3.5
https://hg.python.org/cpython/rev/16accac4b2f6
msg256511 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-12-16 05:33
Thanks Berker
History
Date User Action Args
2022-04-11 14:58:14adminsetgithub: 67976
2015-12-16 05:33:32martin.pantersetstatus: open -> closed
resolution: fixed
messages: + msg256511

stage: commit review -> resolved
2015-12-16 05:27:25python-devsetnosy: + python-dev
messages: + msg256510
2015-12-16 03:25:00berker.peksagsetnosy: + berker.peksag

messages: + msg256506
stage: patch review -> commit review
2015-12-08 07:56:38martin.pantersetfiles: + bad_address.patch
versions: + Python 3.6
messages: + msg256102

keywords: + patch
stage: patch review
2015-12-07 11:09:35martin.pantersetmessages: + msg256057
2015-12-07 06:58:39martin.pantersetmessages: + msg256052
2015-12-07 06:25:16martin.pantersetnosy: + martin.panter
messages: + msg256051
2015-11-20 00:41:08Steve Harrissetnosy: + Steve Harris
messages: + msg254945
2015-03-27 15:12:56vstinnersetmessages: + msg239411
2015-03-27 15:01:45r.david.murraysetnosy: + r.david.murray
messages: + msg239409
2015-03-27 13:07:58ranieresetmessages: + msg239398
2015-03-27 12:43:04vstinnersetmessages: + msg239392
2015-03-27 12:41:32vstinnersetnosy: + vstinner
messages: + msg239391
2015-03-27 12:33:52vstinnersettitle: test_bad_address fails -> test_urllib2_localnet.test_bad_address fails: OSError not raised by urlopen
2015-03-27 12:18:21ranierecreate