Issue7197
Created on 2009-10-24 11:13 by pitrou, last changed 2009-11-11 15:42 by pitrou.
| File name |
Uploaded |
Description |
Edit |
Remove |
|
testrunner.patch
|
pitrou,
2009-10-24 11:49
|
|
|
|
|
msg94406 - (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2009-10-24 11:13 |
|
When run under Windows in verbose mode ("python -m test.regrtest -v
test_multiprocessing"), most tests in test_multiprocessing fail with a
recursion limit error.
The explanation is that most tests are written in the following manner:
class _TestArray(BaseTestCase):
[...]
def test_array(self, raw=False):
[...]
p = self.Process(target=self.f, args=(arr,))
Running a Process under Windows involved pickling it to send it to the
child process. This also pickles the `target` function, which here is a
method of an unittest instance. This consequently pickles the unittest
instance, which has a reference to the unittest runner, which has a
reference to a unittest.runner._WritelnDecorator instance.
The infinite recursion occurs when unpickling the _WritelnDecorator
instance, because the `stream` attribute is not restored when calling
__new__, and the __getattr__ method then recurses when trying to return
`getattr(self.stream,attr)`.
I see two possible resolutions:
- make unittest.runner._WritelnDecorator properly (un)pickleable
- change all tests in test_multiprocessing to avoid pickling instances
of unittest.TestCase
The former is simpler and probably more future-proof than the latter.
(NB: in non-verbose mode, test.support uses a custom test runner which
doesn't involve the _WritelnDecorator)
Appendix: here is a traceback example (noticed on the newgil branch but
it really happens on stock trunk and py3k):
test_notify (test.test_multiprocessing.WithProcessesTestCondition) ...
Traceback
(most recent call last):
File "<string>", line 1, in <module>
File "Z:\py3k\newgil\lib\multiprocessing\forking.py", line 339, in main
self = load(from_parent)
File "Z:\py3k\newgil\lib\pickle.py", line 1365, in load
encoding=encoding, errors=errors).load()
File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
return getattr(self.stream,attr)
[... snipped ...]
RuntimeError: maximum recursion depth exceeded while calling a Python
object
|
|
msg94407 - (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2009-10-24 11:33 |
|
Amusingly, there is a Test_TextTestRunner but it's not run by
test_unittest....
|
|
msg94408 - (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2009-10-24 11:49 |
|
Here is a patch.
|
|
msg95129 - (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2009-11-10 20:29 |
|
Ok, if nobody objects, I will commit it since it's simple enough and
blocks any attempt at diagnosing test_multiprocessing under Windows.
|
|
msg95130 - (view) |
Author: Jesse Noller (jnoller) |
Date: 2009-11-10 20:31 |
|
Please do; I've been swamped - I'm sorry
|
|
msg95141 - (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2009-11-11 15:42 |
|
This has been committed to trunk (r76196) and py3k (r76198).
I'm reluctant to backport it to 2.6/3.1 because unittest has changed a
lot in-between, and the old version doesn't have a lot of tests. Please
do if you think it's useful :)
|
|
| Date |
User |
Action |
Args |
| 2009-11-11 15:42:47 | pitrou | set | status: open -> closed resolution: fixed messages:
+ msg95141
|
| 2009-11-10 20:31:23 | jnoller | set | messages:
+ msg95130 |
| 2009-11-10 20:29:45 | pitrou | set | messages:
+ msg95129 |
| 2009-10-24 11:49:15 | pitrou | set | files:
+ testrunner.patch keywords:
+ patch messages:
+ msg94408
stage: patch review |
| 2009-10-24 11:33:15 | pitrou | set | messages:
+ msg94407 |
| 2009-10-24 11:13:44 | pitrou | create | |
|