It seems a little negligent that test.support has no tests. The test.test_support name is available in py3k and backporting issues to 2.7 shouldn't be a problem (rare chance something does happen it can be handling manually). There doesn't seem to be any reason not to make sure that proper unit testing is being done short of just laziness on our parts.

And if this does occur it should probably be listed as an essential test in regrtest.

I should note that I am not suggesting that test.support suddenly be considered a module with a stable API that the public can use. Simply that we take the proper measures to make sure the code in there does what is expected.

Attached is a patch which tests about half the API.
Certain functions are not documented and/or it's not clear what they do, for others it seems unlikely they are actually used nowadays (although I haven't checked).
Also, it probably makes sense to get rid of some functions which look very old or completely useless.

Giampaolo, FYI I'm also working (issue 11015) in parallel on documenting some of the undocumented functions in Doc/library/test.rst that look useful to me. I think we can collaborate here to make sure they're both documented and tested.

I'll review your patch

P.S. see discussion on pydev about the removal of fcmp

Giampaolo,

The patch looks good. Some minor comments below:

* class TestCase(unittest.TestCase): -> why not give the test class a more useful name?
* import_fresh_module is tested similarly to import_module - can't a test be added that it indeed performs its "special sauce"?
* test_DirsOnSysPath: you may want to use assertIn and assertNotIn here. [these were added to Python in 3.1]
* test_python_is_optimized: you may want to use assertIsInstance here.

test_warnings actually includes sanity checks for import_fresh_module - they could be adapted into a separate unit test easily enough.

+        os.mkdir(os.path.join(TESTFN, TESTFN))

Please don't. This will break the day TESTFN becomes a non-trivial path.

+    def test_make_bad_fd(self):
+        fd = support.make_bad_fd()
+        self.assertRaises(OSError, os.write, fd, b"foo")

You should also check that the errno is EBADF.

> * class TestCase(unittest.TestCase): -> why not give the test class a 
> more useful name?

done

> * import_fresh_module is tested similarly to import_module - can't a 
> test be added that it indeed performs its "special sauce"?

Even by reading the source code I'm not sure what import_fresh_module does exactly.
Feel free to write a test for it if you want.

> * test_DirsOnSysPath: you may want to use assertIn and assertNotIn here. [these 
> were added to Python in 3.1]

done

> * test_python_is_optimized: you may want to use assertIsInstance here.

done

> +        os.mkdir(os.path.join(TESTFN, TESTFN))
> Please don't. This will break the day TESTFN becomes a non-trivial path.

done

> You should also check that the errno is EBADF.

Done. I just hope it's gonna be reliable on all platforms.

Can we commit this patch?

I agree that the docs for import_fresh_module are confusing.  The code says there are sanity checks in test_heapq and test_warnings, so that code could presumably be used as a model for someone to develop a more complete stand-alone test (I haven't looked for that code myself).  I'm not sure why we have both import_fresh_module and CleanImport, since they seem to do similar things (but differently!)

I started making comments on other tests and then realized it wasn't much more work to just update them myself.  Extended patch attached.

Gah.  I always check to see if the patch uploaded before I delete my working copy, but this time I didn't :( :(

I guess I'll come back to this during the sprints.

Any news on this? Can you check that your patches apply for latest trunk and commit them?

Alright, since *some* tests are better than no tests at all, I will try to combine the patches currently attached to the issue, port them to the Hg repo and commit.

Brett, what do you mean by "listed as an essential test in regrtest". The regrtest.STDTESTS list?

New changeset be558ad15789 by Eli Bendersky in branch 'default':
Issue #11049: adding some tests to test.support
http://hg.python.org/cpython/rev/be558ad15789

This changeset incorporates Giampaolo's patch with a minor fix in `test.support.python_is_optimized` (which returned '' for False sometimes).

Several buildbot are failing after the commit:

======================================================================
FAIL: test_get_original_stdout (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildslave/3.x.murray-gentoo-wide/build/Lib/test/test_support.py", line 34, in test_get_original_stdout
    self.assertEqual(support.get_original_stdout(), sys.stdout)
AssertionError: <_io.TextIOWrapper name=1 mode='w' encoding='ANSI_X3.4-1968'> != <_io.StringIO object at 0xb715b46c>

----------------------------------------------------------------------

New changeset 0018a28583f4 by Eli Bendersky in branch 'default':
Issue #11049: skip a test that fails on some buildbots
http://hg.python.org/cpython/rev/0018a28583f4

OK, the bots are green again after the last push.

Sorry about that, folks. I tested this on two different machines (Ubuntu & Win XP) before pushing the original commit.

On Fri, Jul 22, 2011 at 22:47, Eli Bendersky <report@bugs.python.org> wrote:

>
> Eli Bendersky <eliben@gmail.com> added the comment:
>
> Brett, what do you mean by "listed as an essential test in regrtest". The
> regrtest.STDTESTS list?
>

Yes

New changeset 0f09b7f90060 by Eli Bendersky in branch 'default':
Issue #11049: added test_support to regrtest.STDTESTS list
http://hg.python.org/cpython/rev/0f09b7f90060

test_support fails in my environment, only when run from an installed Python, due to permissions issues:

======================================================================
ERROR: test_forget (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.3/test/test_support.py", line 59, in test_forget
    support.forget("smtplib")
  File "/usr/local/lib/python3.3/test/support.py", line 248, in forget
    unlink(imp.cache_from_source(source, debug_override=True))
  File "/usr/local/lib/python3.3/test/support.py", line 205, in unlink
    os.unlink(filename)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python3.3/__pycache__/smtplib.cpython-33.pyc'

The issue is similar to #12331, which was recently fixed.

New changeset 1a978892a105 by Eli Bendersky in branch 'default':
Issue #11049: fix test_forget to work on installed Python, by using a temporary module for import/forget
http://hg.python.org/cpython/rev/1a978892a105

test_forget still doesn't work using my installed Python:

test test_support failed -- Traceback (most recent call last):
  File "/usr/local/lib/python3.3/test/test_support.py", line 62, in test_forget
    mod = __import__(TESTFN)
ImportError: No module named '@test_20485_tmp'

New changeset 07d94cf3521e by Eli Bendersky in branch 'default':
Issue #11049: fix test_forget on installed Python - add os.curdir to sys.path
http://hg.python.org/cpython/rev/07d94cf3521e

Vinay, I can't reproduce your problem here (tried installing Python and running from there), but I suspect it's a sys.path issue (you don't have os.curdir on it, I do). Please let me know if it works now.

Eli, test_support is now completing on my system without errors.

Vinay, great news. Thanks.

Can this be closed, David?

Sure.  It doesn't seem likely that I'll get back to it any time soon, and if I do I can open a new issue.