Author pitrou
Recipients jnoller, michael.foord, pitrou
Date 2009-10-24.11:13:42
SpamBayes Score 3.76076e-09
Marked as misclassified No
Message-id <1256382825.55.0.179199170316.issue7197@psf.upfronthosting.co.za>
In-reply-to
Content
When run under Windows in verbose mode ("python -m test.regrtest -v
test_multiprocessing"), most tests in test_multiprocessing fail with a
recursion limit error.

The explanation is that most tests are written in the following manner:

class _TestArray(BaseTestCase):
    [...]
    def test_array(self, raw=False):
        [...]
        p = self.Process(target=self.f, args=(arr,))

Running a Process under Windows involved pickling it to send it to the
child process. This also pickles the `target` function, which here is a
method of an unittest instance. This consequently pickles the unittest
instance, which has a reference to the unittest runner, which has a
reference to a unittest.runner._WritelnDecorator instance.

The infinite recursion occurs when unpickling the _WritelnDecorator
instance, because the `stream` attribute is not restored when calling
__new__, and the __getattr__ method then recurses when trying to return
`getattr(self.stream,attr)`.

I see two possible resolutions:
- make unittest.runner._WritelnDecorator properly (un)pickleable 
- change all tests in test_multiprocessing to avoid pickling instances
of unittest.TestCase

The former is simpler and probably more future-proof than the latter.

(NB: in non-verbose mode, test.support uses a custom test runner which
doesn't involve the _WritelnDecorator)


Appendix: here is a traceback example (noticed on the newgil branch but
it really happens on stock trunk and py3k):

test_notify (test.test_multiprocessing.WithProcessesTestCondition) ...
Traceback
 (most recent call last):
  File "<string>", line 1, in <module>
  File "Z:\py3k\newgil\lib\multiprocessing\forking.py", line 339, in main
    self = load(from_parent)
  File "Z:\py3k\newgil\lib\pickle.py", line 1365, in load
    encoding=encoding, errors=errors).load()
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
  File "Z:\py3k\newgil\lib\unittest\runner.py", line 21, in __getattr__
    return getattr(self.stream,attr)
[... snipped ...]
RuntimeError: maximum recursion depth exceeded while calling a Python
object
History
Date User Action Args
2009-10-24 11:13:45pitrousetrecipients: + pitrou, jnoller, michael.foord
2009-10-24 11:13:45pitrousetmessageid: <1256382825.55.0.179199170316.issue7197@psf.upfronthosting.co.za>
2009-10-24 11:13:43pitroulinkissue7197 messages
2009-10-24 11:13:42pitroucreate