classification
Title: add tests for test.support
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.3
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: eli.bendersky, eric.araujo, ezio.melotti, giampaolo.rodola, ncoghlan, pitrou, python-dev, r.david.murray, sandro.tosi, vinay.sajip
Priority: normal Keywords: easy, patch

Created on 2011-01-28 19:05 by brett.cannon, last changed 2013-01-25 20:09 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
test_support.patch giampaolo.rodola, 2011-01-29 04:12
test_support.patch giampaolo.rodola, 2011-01-29 14:41
test_support.patch r.david.murray, 2011-02-23 04:25 review
unnamed brett.cannon, 2011-07-26 00:42
Messages (30)
msg127322 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-28 19:05
It seems a little negligent that test.support has no tests. The test.test_support name is available in py3k and backporting issues to 2.7 shouldn't be a problem (rare chance something does happen it can be handling manually). There doesn't seem to be any reason not to make sure that proper unit testing is being done short of just laziness on our parts.

And if this does occur it should probably be listed as an essential test in regrtest.
msg127323 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-01-28 19:06
I should note that I am not suggesting that test.support suddenly be considered a module with a stable API that the public can use. Simply that we take the proper measures to make sure the code in there does what is expected.
msg127379 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-01-29 04:12
Attached is a patch which tests about half the API.
Certain functions are not documented and/or it's not clear what they do, for others it seems unlikely they are actually used nowadays (although I haven't checked).
Also, it probably makes sense to get rid of some functions which look very old or completely useless.
msg127381 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-01-29 04:40
Giampaolo, FYI I'm also working (issue 11015) in parallel on documenting some of the undocumented functions in Doc/library/test.rst that look useful to me. I think we can collaborate here to make sure they're both documented and tested.

I'll review your patch

P.S. see discussion on pydev about the removal of fcmp
msg127383 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-01-29 05:22
Giampaolo,

The patch looks good. Some minor comments below:

* class TestCase(unittest.TestCase): -> why not give the test class a more useful name?
* import_fresh_module is tested similarly to import_module - can't a test be added that it indeed performs its "special sauce"?
* test_DirsOnSysPath: you may want to use assertIn and assertNotIn here. [these were added to Python in 3.1]
* test_python_is_optimized: you may want to use assertIsInstance here.
msg127407 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-01-29 12:20
test_warnings actually includes sanity checks for import_fresh_module - they could be adapted into a separate unit test easily enough.
msg127409 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-29 12:45
+        os.mkdir(os.path.join(TESTFN, TESTFN))

Please don't. This will break the day TESTFN becomes a non-trivial path.

+    def test_make_bad_fd(self):
+        fd = support.make_bad_fd()
+        self.assertRaises(OSError, os.write, fd, b"foo")

You should also check that the errno is EBADF.
msg127420 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-01-29 14:41
> * class TestCase(unittest.TestCase): -> why not give the test class a 
> more useful name?

done

> * import_fresh_module is tested similarly to import_module - can't a 
> test be added that it indeed performs its "special sauce"?

Even by reading the source code I'm not sure what import_fresh_module does exactly.
Feel free to write a test for it if you want.

> * test_DirsOnSysPath: you may want to use assertIn and assertNotIn here. [these 
> were added to Python in 3.1]

done

> * test_python_is_optimized: you may want to use assertIsInstance here.

done

> +        os.mkdir(os.path.join(TESTFN, TESTFN))
> Please don't. This will break the day TESTFN becomes a non-trivial path.

done

> You should also check that the errno is EBADF.

Done. I just hope it's gonna be reliable on all platforms.
msg129122 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-02-22 20:27
Can we commit this patch?
msg129155 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-02-23 04:25
I agree that the docs for import_fresh_module are confusing.  The code says there are sanity checks in test_heapq and test_warnings, so that code could presumably be used as a model for someone to develop a more complete stand-alone test (I haven't looked for that code myself).  I'm not sure why we have both import_fresh_module and CleanImport, since they seem to do similar things (but differently!)

I started making comments on other tests and then realized it wasn't much more work to just update them myself.  Extended patch attached.
msg130651 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-03-12 02:08
Gah.  I always check to see if the patch uploaded before I delete my working copy, but this time I didn't :( :(

I guess I'll come back to this during the sprints.
msg135260 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-05-06 05:49
Any news on this? Can you check that your patches apply for latest trunk and commit them?
msg140928 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-07-23 05:28
Alright, since *some* tests are better than no tests at all, I will try to combine the patches currently attached to the issue, port them to the Hg repo and commit.
msg140929 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-07-23 05:47
Brett, what do you mean by "listed as an essential test in regrtest". The regrtest.STDTESTS list?
msg140930 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-07-23 05:51
New changeset be558ad15789 by Eli Bendersky in branch 'default':
Issue #11049: adding some tests to test.support
http://hg.python.org/cpython/rev/be558ad15789
msg140931 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-07-23 05:53
This changeset incorporates Giampaolo's patch with a minor fix in `test.support.python_is_optimized` (which returned '' for False sometimes).
msg140934 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2011-07-23 06:26
Several buildbot are failing after the commit:

======================================================================
FAIL: test_get_original_stdout (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/buildslave/3.x.murray-gentoo-wide/build/Lib/test/test_support.py", line 34, in test_get_original_stdout
    self.assertEqual(support.get_original_stdout(), sys.stdout)
AssertionError: <_io.TextIOWrapper name=1 mode='w' encoding='ANSI_X3.4-1968'> != <_io.StringIO object at 0xb715b46c>

----------------------------------------------------------------------
msg140977 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-07-23 12:01
New changeset 0018a28583f4 by Eli Bendersky in branch 'default':
Issue #11049: skip a test that fails on some buildbots
http://hg.python.org/cpython/rev/0018a28583f4
msg140980 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-07-23 12:22
OK, the bots are green again after the last push.

Sorry about that, folks. I tested this on two different machines (Ubuntu & Win XP) before pushing the original commit.
msg141132 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2011-07-26 00:42
On Fri, Jul 22, 2011 at 22:47, Eli Bendersky <report@bugs.python.org> wrote:

>
> Eli Bendersky <eliben@gmail.com> added the comment:
>
> Brett, what do you mean by "listed as an essential test in regrtest". The
> regrtest.STDTESTS list?
>

Yes
msg141247 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-07-27 17:23
New changeset 0f09b7f90060 by Eli Bendersky in branch 'default':
Issue #11049: added test_support to regrtest.STDTESTS list
http://hg.python.org/cpython/rev/0f09b7f90060
msg141526 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-08-01 18:48
test_support fails in my environment, only when run from an installed Python, due to permissions issues:

======================================================================
ERROR: test_forget (test.test_support.TestSupport)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.3/test/test_support.py", line 59, in test_forget
    support.forget("smtplib")
  File "/usr/local/lib/python3.3/test/support.py", line 248, in forget
    unlink(imp.cache_from_source(source, debug_override=True))
  File "/usr/local/lib/python3.3/test/support.py", line 205, in unlink
    os.unlink(filename)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python3.3/__pycache__/smtplib.cpython-33.pyc'

The issue is similar to #12331, which was recently fixed.
msg141544 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-08-02 03:27
New changeset 1a978892a105 by Eli Bendersky in branch 'default':
Issue #11049: fix test_forget to work on installed Python, by using a temporary module for import/forget
http://hg.python.org/cpython/rev/1a978892a105
msg141584 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-08-02 22:36
test_forget still doesn't work using my installed Python:

test test_support failed -- Traceback (most recent call last):
  File "/usr/local/lib/python3.3/test/test_support.py", line 62, in test_forget
    mod = __import__(TESTFN)
ImportError: No module named '@test_20485_tmp'
msg141593 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-08-03 02:21
New changeset 07d94cf3521e by Eli Bendersky in branch 'default':
Issue #11049: fix test_forget on installed Python - add os.curdir to sys.path
http://hg.python.org/cpython/rev/07d94cf3521e
msg141594 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-08-03 02:22
Vinay, I can't reproduce your problem here (tried installing Python and running from there), but I suspect it's a sys.path issue (you don't have os.curdir on it, I do). Please let me know if it works now.
msg141607 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2011-08-03 11:54
Eli, test_support is now completing on my system without errors.
msg141608 - (view) Author: Eli Bendersky (eli.bendersky) * (Python committer) Date: 2011-08-03 12:06
Vinay, great news. Thanks.
msg180609 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-01-25 19:21
Can this be closed, David?
msg180620 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-01-25 20:09
Sure.  It doesn't seem likely that I'll get back to it any time soon, and if I do I can open a new issue.
History
Date User Action Args
2013-01-25 20:09:50r.david.murraysetstatus: open -> closed

messages: + msg180620
stage: patch review -> resolved
2013-01-25 19:21:51brett.cannonsetstatus: pending -> open
nosy: - brett.cannon
2013-01-25 19:21:38brett.cannonsetstatus: open -> pending
nosy: brett.cannon, vinay.sajip, ncoghlan, pitrou, giampaolo.rodola, ezio.melotti, eric.araujo, r.david.murray, eli.bendersky, sandro.tosi, python-dev
messages: + msg180609
2011-08-03 12:06:47eli.benderskysetmessages: + msg141608
2011-08-03 11:54:09vinay.sajipsetmessages: + msg141607
2011-08-03 02:22:41eli.benderskysetmessages: + msg141594
2011-08-03 02:21:42python-devsetmessages: + msg141593
2011-08-02 22:36:16vinay.sajipsetmessages: + msg141584
2011-08-02 03:27:27python-devsetmessages: + msg141544
2011-08-01 18:48:11vinay.sajipsetnosy: + vinay.sajip
messages: + msg141526
2011-07-30 13:48:49eric.araujosetnosy: + eric.araujo
2011-07-27 17:23:18python-devsetmessages: + msg141247
2011-07-26 00:42:40brett.cannonsetfiles: + unnamed

messages: + msg141132
2011-07-23 12:22:38eli.benderskysetmessages: + msg140980
2011-07-23 12:01:05python-devsetmessages: + msg140977
2011-07-23 06:26:22ezio.melottisetmessages: + msg140934
2011-07-23 05:53:34eli.benderskysetmessages: + msg140931
2011-07-23 05:51:40python-devsetnosy: + python-dev
messages: + msg140930
2011-07-23 05:47:39eli.benderskysetmessages: + msg140929
2011-07-23 05:28:47eli.benderskysetmessages: + msg140928
2011-05-06 05:49:36eli.benderskysetmessages: + msg135260
2011-05-04 19:08:32sandro.tosisetnosy: + sandro.tosi
2011-03-12 03:28:02r.david.murraysetassignee: r.david.murray
nosy: brett.cannon, ncoghlan, pitrou, giampaolo.rodola, ezio.melotti, r.david.murray, eli.bendersky
2011-03-12 02:08:29r.david.murraysetnosy: brett.cannon, ncoghlan, pitrou, giampaolo.rodola, ezio.melotti, r.david.murray, eli.bendersky
messages: + msg130651
2011-02-23 04:25:38r.david.murraysetfiles: + test_support.patch
nosy: brett.cannon, ncoghlan, pitrou, giampaolo.rodola, ezio.melotti, r.david.murray, eli.bendersky
messages: + msg129155
2011-02-22 20:27:55giampaolo.rodolasetnosy: brett.cannon, ncoghlan, pitrou, giampaolo.rodola, ezio.melotti, r.david.murray, eli.bendersky
messages: + msg129122
2011-02-01 17:05:48ezio.melottisetnosy: + ezio.melotti
2011-01-29 14:41:03giampaolo.rodolasetfiles: + test_support.patch
nosy: brett.cannon, ncoghlan, pitrou, giampaolo.rodola, r.david.murray, eli.bendersky
messages: + msg127420
2011-01-29 12:45:53pitrousetnosy: + pitrou
messages: + msg127409
2011-01-29 12:20:58ncoghlansetnosy: brett.cannon, ncoghlan, giampaolo.rodola, r.david.murray, eli.bendersky
messages: + msg127407
2011-01-29 05:22:07eli.benderskysetnosy: brett.cannon, ncoghlan, giampaolo.rodola, r.david.murray, eli.bendersky
messages: + msg127383
2011-01-29 04:43:23eli.benderskysetnosy: brett.cannon, ncoghlan, giampaolo.rodola, r.david.murray, eli.bendersky
type: behavior
stage: needs patch -> patch review
2011-01-29 04:42:59eli.benderskysetnosy: + ncoghlan
2011-01-29 04:40:54eli.benderskysetnosy: + eli.bendersky
messages: + msg127381
2011-01-29 04:12:17giampaolo.rodolasetfiles: + test_support.patch

nosy: + giampaolo.rodola
messages: + msg127379

keywords: + patch
2011-01-28 19:06:11brett.cannonsetmessages: + msg127323
2011-01-28 19:05:31brett.cannoncreate