The test package is almost to the point where I can run PCbuild\python_d.exe -m unittest discover [Lib/test/](https://github.com/python/cpython/blob/main/Lib/test) "test_*.py" and get a useful result, the only thing that still blows up is multibytecodec_support.py. I had not previously noticed this problem due to the way support.open_urlresource works, requiring the 'urlfetch' resource if and only if the requested file has not already been fetched before and stored in the Lib/test/data dir. multibytecodec_support.TestBase_Mapping blows things up because it uses init instead of setUp to try opening the needed file, which causes ResourceDenied to be raised at class creation time rather than test run time, which unittest can't handle. The attached patch fixes this (at the expense of opening and closing the file once per test method, rather than once per class) as well as converting the test_codecmaps* scripts from test_main to unittest.main.

What is a purpose of support.use_resources = ['urlfetch']? I don't see any effect.

It enables the resource when the tests are run via unittest. (That is, it is equivalent to specifying '-u urlfetch' when running regrtest.)

But I don't see any effect.

$ ./python -m test.regrtest -v test_codecmaps_tw
== CPython 3.3.2+ (3.3:359002d4370d+, Jul 14 2013, 15:34:54) [GCC 4.6.3]
==   Linux-3.8.0-27-generic-i686-with-debian-wheezy-sid little-endian
==   /home/serhiy/py/cpython-3.3/build/test_python_6295
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1)
[1/1] test_codecmaps_tw
test_errorhandle (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test.test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_errorhandle (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test.test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"

$ ./python -m unittest discover -v Lib/test/ "test_codecmaps_tw.py"
test_errorhandle (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test_codecmaps_tw.TestBIG5Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_errorhandle (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_file (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"
test_mapping_supplemental (test_codecmaps_tw.TestCP950Map) ... skipped "Use of the 'urlfetch' resource not enabled"

I'm sorry, I misspoke. It only works when you run the test file as command:

./python Lib/test/test_codecmaps_cn.py

(which uses unittest.main, thus my error in calling it 'via unittest').

So, the fact that the tests don't run if discovery is used is still a bug. I'm not sure how we fix it, given the nature of the resource.

R. David Murray wrote:

So, the fact that the tests don't run if discovery is used is still a
bug. I'm not sure how we fix it, given the nature of the resource.

My preferred solution short of adding resource management to unittest is to add a "regrtest_run" flag to test.support, defaulting to False and set to True by regrtest (like verbose, use_resources, etc.). Then we can replace the sys._getframe hack in support.requires with a simple check of support.regrtest_run. If it's False, return. This way, running the test module directly or via discovery will work the same way: all resources enabled.

+1. You should probably open a new issue for that proposal.

Done, bpo-18492.

I'll submit and updated patch here when (if :)) that one is applied.

I'm also looking into adding resource handling to unittest itself and it is going pretty well; I hope to have a patch ready for review for that soon.

@zach just a reminder that you've committed bpo-18492 in case you'd forgotten this.

New changeset 27feb652d3ad by Zachary Ware in branch '3.4':
Issue bpo-18258: Fix test discovery for test_codecmaps_*.
http://hg.python.org/cpython/rev/27feb652d3ad

New changeset b08921c7d1ec by Zachary Ware in branch 'default':
Issue bpo-18258: Fix test discovery for test_codecmaps_*.
http://hg.python.org/cpython/rev/b08921c7d1ec

Not forgotten, just hadn't gotten to on the list yet.