classification
Title: set up and use VM for net access in the test suite
Type: enhancement Stage:
Components: Tests Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, ammar2, benjamin.peterson, berker.peksag, christian.heimes, dstufft, ezio.melotti, georg.brandl, pitrou, python-dev, r.david.murray
Priority: normal Keywords:

Created on 2014-10-15 22:27 by georg.brandl, last changed 2017-09-04 21:11 by christian.heimes.

Messages (14)
msg229505 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-10-15 22:27
Our test suite currently contacts a range of hosts for different "online" tests, including python.org hosts.  This has led to sudden breakages in the past, e.g. when www.python.org changed URLs or switched to https by default.

It would probably a good idea to consolidate as much as possible into accessing a new VM called something like "testsuite.python.org", where the necessary services can be kept running regardless of the necessities of actual python.org services.

First step to do this is to compile a list of tests that could use the VM, and a list of services that need to be running.
msg229978 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2014-10-25 02:56
There are two disabled tests in Lib/test/test_urllib2net.py:

* OtherNetworkTests.test_cnri:

    # XXX Following test depends on machine configurations that are internal
    # to CNRI.  Need to set up a public server with the right authentication
    # configuration for test purposes.

* AuthTests.test_basic_auth:

    ## Disabled at the moment since there is no page under python.org which
    ## could be used to HTTP authentication.
msg230024 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-10-26 08:57
OK, a hopefully complete overview of the network-using tests and the services
they use, checked by tracing connect() calls during a test run:

* test_codecencodings_*, test_normalization, test_ucn: some Unicode-related data files from

  unicode.org:80
  icu-project.org:80
  people.freebsd.org:80

  Most of these files are not expected to change; they could be copied to our
  new host.  Some are dependent on the unidata version, new versions would have
  to be copied each time after updating the unicodedata database.

* test_httplib: checks HTTPS with

  svn.python.org:443

  This would require a basic HTTPS setup on the host.

* test_imaplib: checks IMAP protocol with

  cyrus.andrew.cmu.edu:143
  cyrus.andrew.cmu.edu:993

  Requires an IMAP server on the host that allows unencrypted and encrypted
  connections (both with STARTTLS and with IMAPS port) and speaks very basic
  IMAP (login then quit).

* test_nntplib: checks NNTP protocol with

  news.trigofacile.com:119
  nntp.aioe.org:563

  Requires an NNTP server on the host that allows unencrypted and encrypted
  connections (both with STARTTLS and on the NNTPS port) and has at least one
  group with some messages.

* test_smtpnet: checks SMTP protocol encryption with

  smtp.gmail.com:25
  smtp.gmail.com:465

  Requires an SMTP server that allows encrypted connections (both with STARTTLS
  and on the SMTPS port) and speaks very basic SMTP (just EHLO and then QUIT).

* test_ssl: checks encrypted connections to

  svn.python.org:443
  sha256.tbs-internet.com:443

  The svn.python.org is probably a basic HTTPS setup.  The second host is
  apparently used for especially checking SHA256.

* test_socket: makes some connection to

  example.com

  while resolving the IDNA name испытание.python.org (which is an alias).

* test_timeout: checks TCP timeout with

  blackhole.snakebite.net:56666
  whitehole.snakebite.net:56667
  www.python.org:80

  The first two are testing timeout with a connection that silently drops
  packets.  It is set up using some packet filtering rules for the two ports.

* test_urllibnet and test_urllib2net: various checks of HTTP, FTP protocols on

  example.com:80
  www.google.com:80
  bit.ly:80
  docs.python.org:443
  imdb.com:80
  ftp.debian.org:21
  ftp.mirror.nl:21
  gatekeeper.research.compaq.com:21

  Most of them are probably fine with a basic readonly FTP server that servers a
  few files and a basic HTTP server.  Some check special cases with broken
  servers (like imdb.com) that might be a little harder to reproduce.

  The two disabled tests mentioned by Berker need HTTP Basic Auth enabled
  on some URL.

* test_xmlrpc_net: checks XML-RPC protocol with

  buildbot.python.org:80

  With the current test, serving a static XML page at a certain URL is
  sufficient: it makes only one request.  Another possibility is setting up a
  small XML-RPC server on a dedicated port.
msg230513 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-11-02 18:14
Note we now have pythontest.net set up.
msg230532 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-03 10:21
Nice! How do you plan to organize its setup?
msg230535 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-11-03 13:34
Why a separate domain rather than a subdomain?  I'm not objecting, but I am curious.
msg230538 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-11-03 14:58
It is configured using salt, see https://github.com/python/psf-salt/blob/master/salt/pythontest/init.sls.

A separate domain just makes it easier to do whatever we need with it without needing to worry about getting confused between live sites and test sites.
msg230569 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-11-03 23:59
Not to mention possible cookie tests, I suppose.
(I would personally have preferred something like "comfychair.net", but I guess pythontest.net is ok too :-))
msg230704 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2014-11-05 18:50
I created a repository https://hg.python.org/pythontestdotnet, which is cloned on the server.
msg230714 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-11-05 21:18
4985375db40f takes care of test_httplib.
msg230739 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-11-06 13:38
New changeset 39536b377241 by Georg Brandl in branch '3.4':
#22650: test suite: load Unicode test data files from www.pythontest.net
https://hg.python.org/cpython/rev/39536b377241
msg230743 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-11-06 13:57
New changeset 0af36ea1d010 by Georg Brandl in branch '2.7':
#22650: test suite: load Unicode test data files from www.pythontest.net
https://hg.python.org/cpython/rev/0af36ea1d010
msg298626 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2017-07-19 01:47
Opened up https://github.com/python/cpython/pull/2755/ in order to address https://bugs.python.org/issue30883

This changes the ftp servers from debian.org to pythontest.net
msg301263 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-04 21:11
FYI, sha256.tbs-internet.com is no longer used in tests.
History
Date User Action Args
2017-09-04 21:11:37christian.heimessetnosy: + christian.heimes
messages: + msg301263
2017-07-19 01:47:00ammar2setnosy: + ammar2
messages: + msg298626
2014-11-06 13:57:05python-devsetmessages: + msg230743
2014-11-06 13:38:53python-devsetnosy: + python-dev
messages: + msg230739
2014-11-05 21:18:53georg.brandlsetmessages: + msg230714
2014-11-05 18:50:34benjamin.petersonsetmessages: + msg230704
2014-11-03 23:59:07pitrousetnosy: + pitrou
messages: + msg230569
2014-11-03 14:58:49dstufftsetmessages: + msg230538
2014-11-03 13:34:53r.david.murraysetmessages: + msg230535
2014-11-03 10:21:35georg.brandlsetmessages: + msg230532
2014-11-02 18:14:32benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg230513
2014-10-26 08:57:03georg.brandlsetmessages: + msg230024
2014-10-25 02:56:22berker.peksagsetmessages: + msg229978
2014-10-16 12:21:51Arfreversetnosy: + Arfrever
2014-10-16 10:37:39berker.peksagsetnosy: + berker.peksag
2014-10-16 03:12:07r.david.murraysetnosy: + r.david.murray
2014-10-15 22:27:08georg.brandlcreate