classification
Title: test_timeout failure when system on IPv4 10.x.x.x subnet
Type: behavior Stage: commit review
Components: Library (Lib), Tests Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: trent Nosy List: Nam.Nguyen, bbrazil, flox, ned.deily, pitrou, python-dev, trent
Priority: normal Keywords: patch

Created on 2012-07-07 16:27 by flox, last changed 2012-08-21 01:45 by trent. This issue is now closed.

Files
File name Uploaded Description Edit
test_timeout.patch trent, 2012-08-18 23:53 review
Messages (15)
msg164906 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-07 16:27
I have this repeatable failure on OSX.


$ time ./python.exe -m test.regrtest -uall -W test_timeout
[1/1] test_timeout
testBlockingThenTimeout (test.test_timeout.CreationTestCase) ... ok
testFloatReturnValue (test.test_timeout.CreationTestCase) ... ok
testObjectCreation (test.test_timeout.CreationTestCase) ... ok
testRangeCheck (test.test_timeout.CreationTestCase) ... ok
testReturnType (test.test_timeout.CreationTestCase) ... ok
testTimeoutThenBlocking (test.test_timeout.CreationTestCase) ... ok
testTypeCheck (test.test_timeout.CreationTestCase) ... ok
testAcceptTimeout (test.test_timeout.TCPTimeoutTestCase) ... ok
testConnectTimeout (test.test_timeout.TCPTimeoutTestCase) ... ERROR
testRecvTimeout (test.test_timeout.TCPTimeoutTestCase) ... ok
testSend (test.test_timeout.TCPTimeoutTestCase) ... ok
testSendall (test.test_timeout.TCPTimeoutTestCase) ... ok
testSendto (test.test_timeout.TCPTimeoutTestCase) ... ok
testRecvfromTimeout (test.test_timeout.UDPTimeoutTestCase) ... ok

======================================================================
ERROR: testConnectTimeout (test.test_timeout.TCPTimeoutTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./Lib/test/test_timeout.py", line 148, in testConnectTimeout
    self._sock_operation(1, 0.001, 'connect', addr)
  File "./Lib/test/test_timeout.py", line 119, in _sock_operation
    method(*args)
PermissionError: [Errno 13] Permission denied

----------------------------------------------------------------------
Ran 14 tests in 9.406s

FAILED (errors=1)
test test_timeout failed
1 test failed:
    test_timeout
[105213 refs]

real	0m9.907s
user	0m0.432s
sys	0m0.038s
msg164914 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-07-07 17:24
What version of OS X?
msg164946 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-07 21:56
happened on 10.6, 64 bits, with a fresh build.

== CPython 3.3.0b1 (default:5b71f5891c54, Jul 7 2012, 17:47:30) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
==   Darwin-10.8.0-i386-64bit little-endian
msg164948 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-07 22:02
probably a temporary network issue: i cannot reproduce it now.
msg164949 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-07 22:06
This was this test which was failing:


    def testConnectTimeout(self):
        # Choose a private address that is unlikely to exist to prevent
        # failures due to the connect succeeding before the timeout.
        # Use a dotted IP address to avoid including the DNS lookup time
        # with the connect time.  This avoids failing the assertion that
        # the timeout occurred fast enough.
        addr = ('10.0.0.0', 12345)
        with support.transient_internet(addr[0]):
            self._sock_operation(1, 0.001, 'connect', addr)


I suspect a conflicting configuration of the EuroPython network ;)
Let's close it.
msg164953 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-07-07 23:20
Actually this is a real test case error.  The problem is observed when the system under test is connected to subnet "10.0.x.x".  The requested address (10.0.0.0) is then a broadcast address for the subnet.  On OS X and (I believe) BSD networking systems, the connect system call checks for this (see man 3 connect) and is documented to fail with:

   [EACCES]  The destination address is a broadcast address and the
             socket option SO_BROADCAST is not set.

When running on a testing Debian Linux 3.2.0 kernel in a VM, the same connect fails with: Errno 113] No route to host, which results in the test being skipped with:
Resource '10.0.0.0' is not available

The test could be changed to pick a (hopefully unused) non-broadcast address or support.transient_internet could add EACCES to its default_errnos, possibly just for BSD/OSX platforms.
msg164974 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-08 08:13
actually, this is the mapping table I have, when it fails with [Errno 13] Permission denied:


$ ifconfig |grep inet
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 
	inet 127.0.0.1 netmask 0xff000000 
	inet 10.20.24.208 netmask 0xfffff000 broadcast 10.20.31.255

$ netstat -r
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            10.20.16.1         UGSc           44        0     en1
10.20.16/20        link#6             UCS            11        0     en1
10.20.16.1         0:c:42:7:1d:80     UHLWI          50      198     en1   1200
10.20.20.29        3c:d0:f8:79:2c:78  UHLWI           0        0     en1    903
10.20.20.77        e4:ce:8f:3c:8f:c8  UHLWI           0        0     en1    816
10.20.20.170       4:54:53:6d:cc:54   UHLWI           0        0     en1    946
10.20.24.3         0:26:bb:6:58:31    UHLWI           0        0     en1   1015
10.20.24.191       3c:d0:f8:a5:76:dc  UHLWI           0        0     en1   1021
10.20.24.208       localhost          UHS             0        0     lo0
10.20.25.89        0:26:8:73:e7:ab    UHLWI           0        0     en1   1190
10.20.25.159       18:3d:a2:10:5e:24  UHLWI           0        0     en1   1146
10.20.25.219       ac:72:89:2e:c9:17  UHLWI           0       78     en1   1142
10.20.29.78        0:1f:3b:3a:6c:8f   UHLWI           0        0     en1   1025
10.20.31.17        68:a8:6d:80:70:de  UHLWI           0        0     en1   1080
127                localhost          UCS             0        0     lo0
localhost          localhost          UH              3   106290     lo0
169.254            link#6             UCS             0        0     en1

Internet6:
Destination        Gateway            Flags         Netif Expire
localhost          localhost          UH              lo0
fe80::%lo0         localhost          Uc              lo0
localhost          link#1             UHL             lo0
fe80::%en0         link#4             UC              en0
ff01::             localhost          Um              lo0
ff02::             localhost          UmC             lo0
ff02::             link#4             UmC             en0
ff02::fb           link#4             UHmLW           en0
msg164977 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2012-07-08 08:45
Something like 127.42.42.42 might be more reliable, because it does not depend on the LAN configuration, only on the local loopback settings.
It is in 127.0.0.0/8 but not in 127.0.0.0/32.
Tested OK here.


127.0.0.0/8 - This block is assigned for use as the Internet host
   loopback address.  A datagram sent by a higher-level protocol to an
   address anywhere within this block loops back inside the host.  This
   is ordinarily implemented using only 127.0.0.1/32 for loopback.  As
   described in [RFC1122], Section 3.2.1.3, addresses within the entire
   127.0.0.0/8 block do not legitimately appear on any network anywhere.

http://tools.ietf.org/html/rfc5735#section-3
msg164983 - (view) Author: Brian Brazil (bbrazil) * Date: 2012-07-08 09:35
I don't think anything in 127/8 is going to work, all Debian/Ubuntu systems I'm aware of have all of 127/8 equivalent to 127.0.0.1.

Maybe 192.0.2.0/24 per RFC 5737?

A better solution might be to chose some host/port on a python.org server and set it to DROP tcp syn packets.
msg166313 - (view) Author: Nam Nguyen (Nam.Nguyen) * Date: 2012-07-24 17:09
Using a remote server such as python.org is a good idea when you have internet connection. When you don't, though, how will the test run?
msg166315 - (view) Author: Brian Brazil (bbrazil) * Date: 2012-07-24 17:18
There's already existing infrastructure not to run certain tests when
there's no internet connection, no ipv6, no tk etc.
msg168542 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-08-18 23:53
Proposed patch attached.
msg168546 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-08-19 05:10
The patch looks good to me.  Thanks for setting up the infrastructure to allow reproducible testing!
msg168726 - (view) Author: Roundup Robot (python-dev) Date: 2012-08-21 01:41
New changeset 9c22222af1f9 by Trent Nelson in branch '3.2':
Issue #15285: Refactor connect timeout test in test_timeout.
http://hg.python.org/cpython/rev/9c22222af1f9

New changeset b86f3af4746c by Trent Nelson in branch 'default':
Issue #15285: Refactor connect timeout test in test_timeout.
http://hg.python.org/cpython/rev/b86f3af4746c
msg168727 - (view) Author: Trent Nelson (trent) * (Python committer) Date: 2012-08-21 01:45
Fixed in 3.2:

changeset:   78690:9c22222af1f9
branch:      3.2
parent:      78686:8600ae913b63
user:        Trent Nelson <trent@trent.me>
date:        Mon Aug 20 21:22:59 2012 -0400
summary:     Issue #15285: Refactor connect timeout test in test_timeout.

Fixed in 3.x:


changeset:   78691:b86f3af4746c
tag:         tip
parent:      78689:f8eabfed9a1d
parent:      78690:9c22222af1f9
user:        Trent Nelson <trent@trent.me>
date:        Mon Aug 20 21:40:21 2012 -0400
summary:     Issue #15285: Refactor connect timeout test in test_timeout.
History
Date User Action Args
2012-08-21 01:45:03trentsetstatus: open -> closed
resolution: fixed
messages: + msg168727
2012-08-21 01:41:29python-devsetnosy: + python-dev
messages: + msg168726
2012-08-21 01:13:42trentsetassignee: trent
2012-08-19 05:10:36ned.deilysetmessages: + msg168546
stage: needs patch -> commit review
2012-08-18 23:53:48trentsetfiles: + test_timeout.patch

nosy: + trent
messages: + msg168542

keywords: + patch
2012-07-24 17:18:46bbrazilsetmessages: + msg166315
2012-07-24 17:09:46Nam.Nguyensetnosy: + Nam.Nguyen
messages: + msg166313
2012-07-08 09:35:25bbrazilsetnosy: + bbrazil
messages: + msg164983
2012-07-08 08:45:14floxsetmessages: + msg164977
title: test_timeout failure when system on IPv4 10.0.x.x subnet -> test_timeout failure when system on IPv4 10.x.x.x subnet
2012-07-08 08:13:30floxsetmessages: + msg164974
2012-07-07 23:20:42ned.deilysetstatus: closed -> open

assignee: ronaldoussoren -> (no value)
components: + Tests, - Macintosh
title: test_timeout failure on OSX -> test_timeout failure when system on IPv4 10.0.x.x subnet
nosy: + pitrou, - ronaldoussoren
versions: + Python 2.7, Python 3.2
messages: + msg164953
resolution: works for me -> (no value)
stage: needs patch
2012-07-07 22:06:40floxsetstatus: pending -> closed

messages: + msg164949
2012-07-07 22:02:26floxsetstatus: open -> pending
resolution: works for me
messages: + msg164948
2012-07-07 21:56:40floxsetmessages: + msg164946
2012-07-07 17:24:55ned.deilysetnosy: + ned.deily
messages: + msg164914
2012-07-07 16:50:33floxsetassignee: ronaldoussoren

components: + Macintosh
nosy: + ronaldoussoren
2012-07-07 16:27:03floxcreate