This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_ftplib timeouts
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, gregory.p.smith, neologix, pitrou, python-dev
Priority: normal Keywords: easy, patch

Created on 2013-08-20 21:01 by pitrou, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
support_host.patch pitrou, 2013-08-21 08:58 review
Messages (14)
msg195717 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-20 21:01
Some ftplib tests sometimes time out. Seem they seem to try to connect to "localhost" (rather than "127.0.0.1"), I was wondering if this could be a DNS issue and if we should resolve "localhost" in advance like Charles-François did for some other tests (or simply hardcode "127.0.0.1", actually).

http://buildbot.python.org/all/builders/x86%20Windows7%203.x/builds/7060/steps/test/logs/stdio

(example:

======================================================================
ERROR: testTimeoutDefault (test.test_ftplib.TestTimeouts)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\test_ftplib.py", line 953, in testTimeoutDefault
    ftp = ftplib.FTP("localhost")
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\ftplib.py", line 115, in __init__
    self.connect(host)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\ftplib.py", line 150, in connect
    source_address=self.source_address)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\socket.py", line 454, in create_connection
    raise err
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\socket.py", line 445, in create_connection
    sock.connect(sa)
socket.timeout: timed out

)
msg195750 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-21 07:25
> Some ftplib tests sometimes time out. Seem they seem to try to connect to "localhost" (rather than "127.0.0.1"), I was wondering if this could be a DNS issue and if we should resolve "localhost" in advance like Charles-François did for some other tests (or simply hardcode "127.0.0.1", actually).

If it only happens on Windows >= 7 buildbots, then it's likely a
consequence of this:
http://serverfault.com/questions/4689/windows-7-localhost-name-resolution-is-handled-within-dns-itself-why

Apparently, since Windows 7, "localhost" doesn't appear in the hosts
file anymore: so, depending on your DNS resolver settings, this could
perfectly explain such timeouts (assuming it's not a firewall issue).

Hard-cording 127.0.0.1 for this test should be OK, because the server
setup explicitly binds an AF_INET socket. We'll probably have to check
the rest of the test suite for similar issues.
msg195752 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-21 08:04
> Hard-cording 127.0.0.1 for this test should be OK, because the server
> setup explicitly binds an AF_INET socket. We'll probably have to check
> the rest of the test suite for similar issues.

Ok, let's do it!
msg195754 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-21 08:58
Here is a patch for default.
msg195766 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-21 10:25
Changing support.HOST from 'localhost' to '127.0.0.1' means that on dual-stack
hosts (I don't think the test suite would run on IPv6-only hosts anyway), the
tests will now always use IPv4, whereas they are currently using either IPv6 or
IPv4 addresses depending on the host's name resolution settings.
For example, on a RHEL6 box:

>>> socket.getaddrinfo('localhost', 0)
[(10, 1, 6, '', ('::1', 0, 0, 0)), (10, 2, 17, '', ('::1', 0, 0, 0)),
(10, 3, 0, '', ('::1', 0, 0, 0)), (2, 1, 6, '', ('127.0.0.1', 0)), (2,
2, 17, '', ('127.0.0.1', 0)), (2, 3, 0, '', ('127.0.0.1', 0))]

So on this host, 'localhost' resolves to '::1'. I think it would be
better to have support.HOST set to socket.getaddrinfo('localhost', 0)[0][4][0]
(sic), so that the test will use the preferred protocol on the target host, and
also that IPv6 is also tested in tests which don't explicitely use
'::1'. But this
means that we should have an support.HOSTv4 for tests that explicitly rely on
IPv4, e.g.  test_ftplib.
msg195770 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-21 10:43
> Changing support.HOST from 'localhost' to '127.0.0.1' means that on
> dual-stack
> hosts (I don't think the test suite would run on IPv6-only hosts
> anyway), the
> tests will now always use IPv4, whereas they are currently using
> either IPv6 or
> IPv4 addresses depending on the host's name resolution settings.

That would only be true if the server we are contacting happens to
listen on both IPv6 and IPv4, right? I don't think we have such a
situation in the test suite. Clients can use create_connection()
for transparent support of IPv6 and IPv4, but servers have to choose
a family when creating their socket.
msg195779 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-21 12:03
> That would only be true if the server we are contacting happens to
> listen on both IPv6 and IPv4, right? I don't think we have such a
> situation in the test suite.

Ah, I thought some code was using getaddrinfo() to select an arbitrary
bind() address for localhost.

That's OK then.
msg195806 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2013-08-21 16:49
LGTM
msg195831 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-08-21 22:39
New changeset 085ba7d85eb2 by Antoine Pitrou in branch 'default':
Issue #18792: Use "127.0.0.1" or "::1" instead of "localhost" as much as possible, since "localhost" goes through a DNS lookup under recent Windows versions.
http://hg.python.org/cpython/rev/085ba7d85eb2
msg195833 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-08-21 22:49
New changeset 7728b073c77c by Antoine Pitrou in branch '3.3':
Issue #18792: Use "127.0.0.1" or "::1" instead of "localhost" as much as possible, since "localhost" goes through a DNS lookup under recent Windows versions.
http://hg.python.org/cpython/rev/7728b073c77c
msg195834 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-08-21 22:58
New changeset 48de8df194d9 by Antoine Pitrou in branch '2.7':
Issue #18792: Use "127.0.0.1" or "::1" instead of "localhost" as much as possible, since "localhost" goes through a DNS lookup under recent Windows versions.
http://hg.python.org/cpython/rev/48de8df194d9
msg195835 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-08-21 22:59
Done, thanks :)
msg288497 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-02-24 01:29
FYI - hardcoding these addresses is now causing me problems as I try to get our test suite passing on IPv6 only hosts.  "localhost" is correct.  

IMNSHO if for some reason a system cannot resolve "localhost" into a correct address, it should be banned from running any form of networking related test. :/

I should not have to write messy conditional code to try and determine which type of socket I will likely need to bind to to determine which format of IP address string I need to supply as the localhost bind address.  That is backwards!
msg288499 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2017-02-24 01:35
https://bugs.python.org/issue29639 opened to track undoing this when appropriate.
History
Date User Action Args
2022-04-11 14:57:49adminsetgithub: 62992
2017-02-24 01:35:26gregory.p.smithsetmessages: + msg288499
2017-02-24 01:29:52gregory.p.smithsetnosy: + gregory.p.smith
messages: + msg288497
2013-08-21 22:59:07pitrousetstatus: open -> closed
resolution: fixed
messages: + msg195835

stage: needs patch -> resolved
2013-08-21 22:58:36python-devsetmessages: + msg195834
2013-08-21 22:49:00python-devsetmessages: + msg195833
2013-08-21 22:39:57python-devsetnosy: + python-dev
messages: + msg195831
2013-08-21 16:49:24neologixsetmessages: + msg195806
2013-08-21 12:03:15neologixsetmessages: + msg195779
2013-08-21 10:43:57pitrousetmessages: + msg195770
2013-08-21 10:25:14neologixsetmessages: + msg195766
2013-08-21 08:58:21pitrousetfiles: + support_host.patch
keywords: + patch
messages: + msg195754
2013-08-21 08:04:11pitrousetkeywords: + easy

messages: + msg195752
2013-08-21 07:25:08neologixsetmessages: + msg195750
2013-08-20 21:01:29pitroucreate