Message298952
I'm not sure that the issue is really fixed. I applied my msg263461 patch and I still see the list of "python" processes growing but then also a little bit decreasing, and some zombi <defunct> processes.
The problem is that it's hard to guess if a growing "ps" list is a bug or not, since we only cleanup things in tearDownClass(), not in tearDown(). So it's ok to keep some child processes between two unit tests.
I wrote a first change to help debugging the issue. My change adds more warnings and calls support.reap_children():
https://github.com/python/cpython/pull/2841
With this PR, the following test fails:
---
haypo@selma$ ./python -u -m test -v --fail-env-changed --match=test.test_multiprocessing_spawn.WithThreadsTestPool.test_release_task_refs --match=test.test_multiprocessing_spawn.WithProcessesTestLogging.test_level test_multiprocessing_spawn
== CPython 3.7.0a0 (heads/master:b364d9f, Jul 24 2017, 11:06:33) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
== Linux-4.11.9-200.fc25.x86_64-x86_64-with-fedora-25-Twenty_Five little-endian
== hash algorithm: siphash24 64bit
== cwd: /home/haypo/prog/python/master/build/test_python_24433
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 load avg: 0.27 [1/1] test_multiprocessing_spawn
test_level (test.test_multiprocessing_spawn.WithProcessesTestLogging) ... ok
Warning -- reap_children() reaped child process 24437
test_release_task_refs (test.test_multiprocessing_spawn.WithThreadsTestPool) ... ok
Warning -- reap_children() reaped child process 24438
Dangling processes: {<Process(Process-2, started daemon)>, <Process(Process-1, started daemon)>}
----------------------------------------------------------------------
Ran 2 tests in 1.369s
OK
Warning -- multiprocessing.process._dangling was modified by test_multiprocessing_spawn
Before: <_weakrefset.WeakSet object at 0x7fdc24f70058>
After: <_weakrefset.WeakSet object at 0x7fdc2a666260>
test_multiprocessing_spawn failed (env changed)
1 test altered the execution environment:
test_multiprocessing_spawn
Total duration: 2 sec
Tests result: ENV CHANGED
--- |
|
Date |
User |
Action |
Args |
2017-07-24 10:43:31 | vstinner | set | recipients:
+ vstinner, pitrou, martin.panter, serhiy.storchaka, davin |
2017-07-24 10:43:31 | vstinner | set | messageid: <1500893011.13.0.321287040227.issue26762@psf.upfronthosting.co.za> |
2017-07-24 10:43:31 | vstinner | link | issue26762 messages |
2017-07-24 10:43:30 | vstinner | create | |
|