classification
Title: Fix test discovery for test_codecmaps*.py
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zach.ware Nosy List: BreamoreBoy, brett.cannon, ezio.melotti, python-dev, r.david.murray, serhiy.storchaka, zach.ware
Priority: normal Keywords: patch

Created on 2013-06-18 21:16 by zach.ware, last changed 2014-07-02 01:26 by zach.ware. This issue is now closed.

Files
File name Uploaded Description Edit
test_codecmaps_discovery.diff zach.ware, 2013-06-18 21:16 test_codecmaps*.py fix, version 1 review
Messages (11)
msg191434 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-06-18 21:16
The test package is almost to the point where I can run `PCbuild\python_d.exe -m unittest discover Lib/test/ "test_*.py"` and get a useful result, the only thing that still blows up is multibytecodec_support.py.  I had not previously noticed this problem due to the way support.open_urlresource works, requiring the 'urlfetch' resource if and only if the requested file has not already been fetched before and stored in the Lib/test/data dir.  multibytecodec_support.TestBase_Mapping blows things up because it uses __init__ instead of setUp to try opening the needed file, which causes ResourceDenied to be raised at class creation time rather than test run time, which unittest can't handle.  The attached patch fixes this (at the expense of opening and closing the file once per test method, rather than once per class) as well as converting the test_codecmaps* scripts from test_main to unittest.main.
msg193229 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-07-17 14:19
What is a purpose of `support.use_resources = ['urlfetch']`? I don't see any effect.
msg193231 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-17 14:51
It enables the resource when the tests are run via unittest.  (That is, it is equivalent to specifying '-u urlfetch' when running regrtest.)
msg193239 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-07-17 16:04
But I don't see any effect.

$ ./python -m test.regrtest -v test_codecmaps_tw
== CPython 3.3.2+ (3.3:359002d4370d+, Jul 14 2013, 15:34:54) [GCC 4.6.3]
==   Linux-3.8.0-27-generic-i686-with-debian-wheezy-sid little-endian
==   /home/serhiy/py/cpython-3.3/build/test_python_6295
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1)
[1/1] test_codecmaps_tw
test_errorhandle (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_errorhandle (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"

$ ./python -m unittest discover -v Lib/test/ "test_codecmaps_tw.py"
test_errorhandle (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_errorhandle (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
msg193240 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-17 16:10
I'm sorry, I misspoke.  It only works when you run the test file as command:

   ./python Lib/test/test_codecmaps_cn.py

(which uses unittest.main, thus my error in calling it 'via unittest').

So, the fact that the tests don't run if discovery is used is still a bug.  I'm not sure how we fix it, given the nature of the resource.
msg193242 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-07-17 17:18
R. David Murray wrote:
> So, the fact that the tests don't run if discovery is used is still a
> bug.  I'm not sure how we fix it, given the nature of the resource.

My preferred solution short of adding resource management to unittest is to add a "regrtest_run" flag to test.support, defaulting to False and set to True by regrtest (like verbose, use_resources, etc.).  Then we can replace the sys._getframe hack in support.requires with a simple check of support.regrtest_run.  If it's False, return.  This way, running the test module directly or via discovery will work the same way: all resources enabled.
msg193243 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-07-17 19:20
+1.  You should probably open a new issue for that proposal.
msg193311 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2013-07-18 15:41
Done, issue18492.

I'll submit and updated patch here when (if :)) that one is applied.

I'm also looking into adding resource handling to unittest itself and it is going pretty well; I hope to have a patch ready for review for that soon.
msg222060 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-01 22:47
@Zach just a reminder that you've committed #18492 in case you'd forgotten this.
msg222063 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-07-02 01:25
New changeset 27feb652d3ad by Zachary Ware in branch '3.4':
Issue #18258: Fix test discovery for test_codecmaps_*.
http://hg.python.org/cpython/rev/27feb652d3ad

New changeset b08921c7d1ec by Zachary Ware in branch 'default':
Issue #18258: Fix test discovery for test_codecmaps_*.
http://hg.python.org/cpython/rev/b08921c7d1ec
msg222065 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-07-02 01:26
Not forgotten, just hadn't gotten to on the list yet.
History
Date User Action Args
2014-07-02 01:26:12zach.waresetstatus: open -> closed
versions: + Python 3.5, - Python 3.3
messages: + msg222065

assignee: zach.ware
resolution: fixed
stage: patch review -> resolved
2014-07-02 01:25:09python-devsetnosy: + python-dev
messages: + msg222063
2014-07-01 22:47:42BreamoreBoysetnosy: + BreamoreBoy
messages: + msg222060
2013-12-17 21:26:38zach.warelinkissue16748 dependencies
2013-07-18 15:41:59zach.waresetmessages: + msg193311
2013-07-17 19:20:10r.david.murraysetmessages: + msg193243
2013-07-17 17:18:54zach.waresetmessages: + msg193242
2013-07-17 16:10:26r.david.murraysetmessages: + msg193240
2013-07-17 16:04:21serhiy.storchakasetmessages: + msg193239
2013-07-17 14:51:36r.david.murraysetnosy: + r.david.murray
messages: + msg193231
2013-07-17 14:19:51serhiy.storchakasetmessages: + msg193229
2013-07-17 11:46:25serhiy.storchakasetnosy: + serhiy.storchaka

stage: patch review
2013-06-18 21:16:21zach.waresettype: enhancement -> behavior
2013-06-18 21:16:03zach.warecreate