Created on 2010-12-08 14:39 by ocean-city, last changed 2014-06-02 16:18 by zach.ware.
|py3k_restore_sys_modules_in_regrtest.patch||ocean-city, 2010-12-08 15:25||review|
|Issue10652.diff||terry.reedy, 2012-07-22 07:53||remove tk* modules after test||review|
|issue10652.v2.diff||zach.ware, 2013-03-15 21:29||import_fresh_module in tcl/tk tests|
|10652_tkfix_27.diff||terry.reedy, 2013-03-16 06:20||partial fix for 2.7|
|issue10652-2.7.diff||zach.ware, 2013-11-04 21:10||Fix for 2.7||review|
|issue10652-2.7-alternate.diff||zach.ware, 2013-11-04 21:13||Alternate fix for 2.7||review|
|msg123609 - (view)||Author: Hirokazu Yamamoto (ocean-city) *||Date: 2010-12-08 14:39|
On official Python3.2 beta1 windows binary, I noticed following command fails. (test_tcl alone won't fail) I couldn't reproduce this on binary built from source without installation. ////////////////////////////////////////////////////////////// C:\Python32>.\python.exe -m test.regrtest -v test___all__ test_tcl (snip) [2/2] test_tcl testCall (test.test_tcl.TclTest) ... ERROR testCallException (test.test_tcl.TclTest) ... ERROR testCallException2 (test.test_tcl.TclTest) ... ERROR testEval (test.test_tcl.TclTest) ... ERROR testEvalException (test.test_tcl.TclTest) ... ERROR testEvalException2 (test.test_tcl.TclTest) ... ERROR testEvalFile (test.test_tcl.TclTest) ... ERROR testEvalFileException (test.test_tcl.TclTest) ... ERROR testGetVar (test.test_tcl.TclTest) ... ERROR testGetVarArray (test.test_tcl.TclTest) ... ERROR testGetVarArrayException (test.test_tcl.TclTest) ... ERROR testGetVarException (test.test_tcl.TclTest) ... ERROR testLoadWithUNC (test.test_tcl.TclTest) ... ERROR testPackageRequireException (test.test_tcl.TclTest) ... ERROR testSetVar (test.test_tcl.TclTest) ... ERROR testSetVarArray (test.test_tcl.TclTest) ... ERROR testUnsetVar (test.test_tcl.TclTest) ... ERROR testUnsetVarArray (test.test_tcl.TclTest) ... ERROR testUnsetVarException (test.test_tcl.TclTest) ... ERROR testFlattenLen (test.test_tcl.TkinterTest) ... ok ====================================================================== ERROR: testCall (test.test_tcl.TclTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python32\lib\test\test_tcl.py", line 25, in setUp self.interp = Tcl() File "C:\Python32\lib\tkinter\__init__.py", line 1768, in Tcl return Tk(screenName, baseName, className, useTk) File "C:\Python32\lib\tkinter\__init__.py", line 1674, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, want objects, useTk, sync, use) _tkinter.TclError: Can't find a usable init.tcl in the following directories: C:/Python32/lib/tcl8.5 C:/lib/tcl8.5 C:/lib/tcl8.5 C:/library C:/library C:/ tcl8.5.2/library C:/tcl8.5.2/library This probably means that Tcl wasn't installed properly. (snip) ---------------------------------------------------------------------- Ran 20 tests in 1.752s FAILED (errors=19) test test_tcl failed -- multiple errors occurred 1 test failed: test_tcl 1 test altered the execution environment: test___all__
|msg123614 - (view)||Author: Hirokazu Yamamoto (ocean-city) *||Date: 2010-12-08 15:09|
test___all__ imports a lot of modules, but I found one of following modules can bring same error. # just import one of these in test_main(test___all__) # import idlelib.AutoComplete # import tkinter.scrolledtext # import tkinter.ttk # import turtle ////////////////////////////////////////////////////// And actually, "import tkinter" is enough.
|msg123615 - (view)||Author: Hirokazu Yamamoto (ocean-city) *||Date: 2010-12-08 15:19|
I think this happens because 1. test___all__.py imports tkinter module, and it imports tkinter/_fix.py 2. _fix.py sets TCL_LIBRARY etc as top level routine 3. regrtest.py resets os.environ after test___all__.py ends. so TCL_LIBRARY gone. 4. test_tcl.py tries to import tkinter module, but it won't be loaded twice, so TCL_LIBRARY won't be set. 5. tcl/tk cannot find its library I think this can be fixed by save & restore sys.modules before & after each tests, so that tkinter module can be loaded again.
|msg123616 - (view)||Author: Hirokazu Yamamoto (ocean-city) *||Date: 2010-12-08 15:25|
How about this patch? Is this kind of *fix* acceptable? # I hope this works.
|msg165888 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-07-19 23:21|
I think this test disabling and failure issue should get some attention. Do the failures happen on non-Windows systems too? With installed 3.3.0b1 on Win7, test___all__, test_tcl, test_tk, test_ttk_textonly, and test_ttk_guionly all run if run first (_tk and _ttk_guionly with -ugui). All modify os.environ, resulting in Warning -- os.environ was modified by test_xxx and all disable at least one of the other tests run afterward. (test___all__ disables but is not disabled itself.) In standard alphabetic running order, test___all__ 'wins'. I would like to add test/test_idle, #15392, but I presume it currently would also not run after test___all__ or any of the others. --- I patched my installation, but without the unneeded temporary (saved_values = self.saved_values is only an optimization for the repeated access in the loop that follows). sys.modules.clear() sys.modules.update(self.saved_modules) del self.saved_modules With this, python -m test -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonly runs and passes all five tests instead of two (test___all__ and test_ttk_textonly). Unfortunately, it causes new failures running the entire test suite: test_concurrent_futures (ok before) test_decimal (ok before) test_email (numberous failures, just one before) I stopped at test_faulthandler, there might be more but this was enough to reject the patch as is. I decided to try the principle of doing the minimum necessary to get the tests to pass. I arrived at the following, which lets all five tests run and pass, at least when run in the given order. try: del sys.modules['tkinter'] del sys.modules['tkinter._fix'] del sys.modules['tkinter.ttk'] del sys.modules['tkinter.test'] del sys.modules['tkinter.test.support'] del sys.modules['tkinter.test.runtktests'] except KeyError: pass The only new failure with python -m test is test_multi-processing, which normally does not even give a warning. [197/368/6] test_multiprocessing Warning -- multiprocessing.process._dangling was modified by test_multiprocessing test test_multiprocessing failed -- Traceback (most recent call last): File "C:\Programs\Python33\lib\test\test_multiprocessing.py", line 1909, in test_mymanager_context_prestarted self.assertEqual(manager._process.exitcode, 0) AssertionError: -15 != 0 This might be an unrelated, intermittant failure. Since the exception above will trigger for nearly all tests (costly), and since the above does not guarantee to delete all tkinter.* modules, and since future tkinter tests might expand to test other subpackages and idle tests will import other subpackages, I tried this in __exit__. if 'tkinter' in sys.modules: sysmod = sys.modules tknames = set() for name in sysmod: if name.startswith('tkinter'): tknames.add(name) for name in tknames: del sysmod[name] So aside from the initial test, the extra time is only used when needed. All five tests run and pass and there are no new failures with python -m test. Before: [368/368/7] test_zlib 316 tests OK. 7 tests failed: test_asyncore test_datetime test_distutils test_email test_ftplib test_import test_tcl 2 tests altered the execution environment: test___all__ test_builtin 43 tests skipped: test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_crypt test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile test_nis test_openpty test_ossaudiodev test_pipes test_poll test_posix test_pty test_pwd test_readline test_resource test_smtpnet test_socketserver test_syslog test_threadsignals test_timeout test_tk test_tools test_ttk_guionly test_urllib2net test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net test_zipfile64 5 skips unexpected on win32: test_gdb test_readline test_tk test_tools test_ttk_guionly After: 315 tests OK. 6 tests failed: test_asyncore test_datetime test_distutils test_email test_ftplib test_import 4 tests altered the execution environment: test___all__ test_builtin test_tcl test_ttk_textonly 43 tests skipped: test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_crypt test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile test_nis test_openpty test_ossaudiodev test_pipes test_poll test_posix test_pty test_pwd test_readline test_resource test_smtpnet test_socketserver test_syslog test_threadsignals test_timeout test_tk test_tools test_ttk_guionly test_urllib2net test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net test_zipfile64 3 skips unexpected on win32: test_gdb test_readline test_tools I am puzzled why one less failure and two fewer unexpected skips means one few test ok. Is that because of the two more warnings? Running with -ugui will cause test_tk and test_ttk_guionly to run instead of skip. So the above patch seems to be a pure win on Windows. What about other systems? And do any of you test experts have anything better?
|msg166057 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-07-21 19:58|
I would like to see my patch (cut and paste from message above) or something like it in the next beta. It fixes the problem (since forever?) of tcl/tk/ttk tests not running properly (at least on windows) as part of the test suite. (I presume the buildbots have extra skips in order to be green.) I am reluctant to apply myself without review simply because I am ignorant enough to miss something that others might see wrong.
|msg166058 - (view)||Author: Georg Brandl (georg.brandl) *||Date: 2012-07-21 20:00|
It would be much better to have an actual patch to review. Then, we need someone who can test it on Windows.
|msg166073 - (view)||Author: R. David Murray (r.david.murray) *||Date: 2012-07-21 21:21|
I wonder if it would be sensible to have test___all__ restore the state of sys.modules after it runs.
|msg166074 - (view)||Author: Antoine Pitrou (pitrou) *||Date: 2012-07-21 21:30|
> I wonder if it would be sensible to have test___all__ restore the > state of sys.modules after it runs. Probably not. I don't think we want to debug the consequences of having duplicate modules lying around by way of old references registered here and there.
|msg166075 - (view)||Author: R. David Murray (r.david.murray) *||Date: 2012-07-21 21:33|
Yeah, we probably do not want to go there, even though technically I think those would be bugs. But bugs that it would only be "nice" to fix, not necessary to fix.
|msg166106 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-07-22 07:53|
I tested this on Windows in my installed 3.3.0b1. Copying the inserted lines to my repository copy (where I cannot test it) gives the attached patch. On 3.2.3, the insertion line would be 1056 instead of 1158.
|msg166107 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2012-07-22 07:58|
Hirokazu's original patch was to restore sys.modules after *every* test, not just sys.modules. That *did* cause problems because (at minimum) of references in importlib. Restoring only after __all__ would only let test_tcl run but not test_tk. And that is assuming alphabetic, not randomized order.
|msg184253 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2013-03-15 20:46|
I was recently bitten by this issue. I've reviewed Terry's patch on Rietveld, and have tested it myself some. It seems to get the job done without getting in the way.
|msg184255 - (view)||Author: Ezio Melotti (ezio.melotti) *||Date: 2013-03-15 20:53|
Would using import_fresh_module() in test_tcl (and others) work?
|msg184262 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2013-03-15 21:29|
It does, in fact. Here's a patch.
|msg184284 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2013-03-16 02:17|
Once I worked around the example.bat problems so repository _tkinter will run for one python version (3.2), Zach's patch seems to solve the test problem as well as mine, and it is better self-contained. In other words, python -m test -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonly runs all five tests with os.environ change the only problem noted. If Ezio or someone else who knows testing better than me agrees that it is the right patch, I am willing to apply it.
|msg184290 - (view)||Author: Ezio Melotti (ezio.melotti) *||Date: 2013-03-16 04:59|
LGTM. The changes on os.environ should be unrelated. I've seen a few other tests (e.g. test_distutils) that change it on Windows, and started writing a patch for it, but I never finished it.
|msg184294 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2013-03-16 06:20|
After editing 2.7 files to match Zach's patch, and also after adding the changes in my patch, and also deleting tcltk directory and rerunning external.bat to rebuild tcltk/ for 2.7 with tcl/tk 8.5.2, the tests still do not all work right. -------------- F:\Python\dev\py27\PCbuild> python_d -m test.regrtest -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_text only test___all__ Warning -- os.environ was modified by test___all__ test_tcl test test_tcl failed -- multiple errors occurred; run in verbose mode for details test_tk test_tk skipped -- tk not available: Can't find a usable init.tcl in the following directories: F:/Python/dev/py27/lib/tcl8.5 F:/Python/dev/py27/lib/tcl8.5 F:/Python/dev/lib/tcl8.5 F:/Python/dev/py27/library F:/P ython/dev/library F:/Python/dev/tcl8.5.2/library F:/Python/tcl8.5.2/library This probably means that Tcl wasn't installed properly. ------ If test_tcl is run first, it works, test_ttk_guionly gives the same message as test_tk, test_ttk_testonly is OK. So the attached patch, 10652_tkfix_27.diff is an improvement, but not a complete fix. I am going to concentrate on 3.x (which means manually delete tcltk and re-compile) and apply if all goes well.
|msg184298 - (view)||Author: Roundup Robot (python-dev)||Date: 2013-03-16 07:01|
New changeset b923234b60cb by Terry Jan Reedy in branch '3.2': Issue # 10652: make tcl/tk tests run after __all__ test, patch by Zachary Ware. http://hg.python.org/cpython/rev/b923234b60cb New changeset 596e8855895e by Terry Jan Reedy in branch '3.3': Issue # 10652: make tcl/tk tests run after __all__ test, patch by Zachary Ware. http://hg.python.org/cpython/rev/596e8855895e New changeset 7c76b70075db by Terry Jan Reedy in branch 'default': #10652 null merge from 3.3 which had wrong issue # http://hg.python.org/cpython/rev/7c76b70075db
|msg189880 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2013-05-23 21:33|
Terry, I just tested your 2.7 patch, and it does work, but the workaround that (I think) we have both been using on 3.x to find the Tcl/Tk .dlls (copying them into PCbuild) doesn't work on 2.7 for some reason. However, if you add ..\tcltk\bin to PATH, which the PCbuild\rt.bat script (called by Tools\buildbot\test(-amd64).bat) does, your patch allows all of the test modules to run properly. I wonder if this has anything to do with issue17883's buildbot problems?
|msg202173 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2013-11-04 21:10|
Having another chance to look at this one, my previous message was incorrect; Terry's patch does not fix the issue for an installed Python 2.7. The only change it needs to work, though, is to replace 'Tkinter' with 'FixTk' in the import_fresh_module call.
|msg202175 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2013-11-04 21:13|
An alternative that works and also removes repeated "Warning -- os.environ was modified by test_*" is to import FixTk at the top of test_support, allowing the environment to be set up and to persist throughout all of the tests. I'm not sure if this is the right way to go about the problem, though.
|msg219522 - (view)||Author: Terry J. Reedy (terry.reedy) *||Date: 2014-06-01 20:12|
Zach, do you have any further thoughts in light of patches pushed since? What do you think is the exact remaining issue?
|msg219601 - (view)||Author: Zachary Ware (zach.ware) *||Date: 2014-06-02 16:18|
As for what's actually wrong here, Hirokazu Yamamoto's diagnosis in msg123615 (adjusted for 2.7) is correct. Either of the last two patches I posted should work to fix this issue, but they're both just band-aids rather than a real, once-and-for-all fix. #20035 (which I need to rewrite again) will be a once-and-for-all fix for 3.5 by getting rid of tkinter._fix, but I'm not sure if such an invasive fix is appropriate for 2.7 and 3.4. I prefer the second band-aid (import FixTk at the top of test_support) just because it's simpler and also prevents the 'os.environ has been changed' warnings. A workaround that doesn't require a patch is to just set TCL_LIBRARY manually in your environment before running the tests, which is how the 3.x buildbots are currently working (see Tools/buildbot/test.bat:4). For the record, I'm not sure why the 3.x fix we came up with earlier in this issue worked, though I suspect it has something to do with _fix being part of the tkinter package. The same patch fails on 2.7 because Tkinter is not a package; FixTk is a standalone module and is thus completely unaffected by support.import_fresh_module('Tkinter'). Fresh-importing FixTk itself works, since it's what we actually need to run.
|2014-06-02 16:18:29||zach.ware||set||messages: + msg219601|
|2014-06-01 20:12:06||terry.reedy||set||assignee: terry.reedy -> |
messages: + msg219522
messages: + msg202175
messages: + msg202173
components: + Tkinter
versions: - Python 3.3, Python 3.4
|2013-06-08 19:05:23||ezio.melotti||set||versions: + Python 2.7|
|2013-05-23 21:33:37||zach.ware||set||messages: + msg189880|
|2013-05-13 06:12:10||terry.reedy||set||versions: - Python 3.2|
messages: + msg184298
messages: + msg184294
|2013-03-16 04:59:31||ezio.melotti||set||assignee: terry.reedy|
messages: + msg184290
stage: commit review -> patch review
stage: patch review -> commit review
messages: + msg184262
versions: + Python 3.4
messages: + msg184253
|2012-07-22 07:58:45||terry.reedy||set||messages: + msg166107|
messages: + msg166106
|2012-07-21 21:33:12||r.david.murray||set||messages: + msg166075|
|2012-07-21 21:30:03||pitrou||set||messages: + msg166074|
|2012-07-21 21:21:01||r.david.murray||set||messages: + msg166073|
|2012-07-21 20:00:41||georg.brandl||set||messages: + msg166058|
messages: + msg166057
versions: + Python 3.3
+ ezio.melotti, r.david.murray, pitrou, terry.reedy, michael.foord|
messages: + msg165888
stage: patch review
keywords: + patch
messages: + msg123616
|2010-12-08 15:19:02||ocean-city||set||messages: + msg123615|
|2010-12-08 15:09:16||ocean-city||set||messages: + msg123614|