Author vstinner
Recipients davin, martin.panter, pitrou, serhiy.storchaka, vstinner
Date 2017-07-24.10:43:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1500893011.13.0.321287040227.issue26762@psf.upfronthosting.co.za>
In-reply-to
Content
I'm not sure that the issue is really fixed. I applied my msg263461 patch and I still see the list of "python" processes growing but then also a little bit decreasing, and some zombi <defunct> processes.

The problem is that it's hard to guess if a growing "ps" list is a bug or not, since we only cleanup things in tearDownClass(), not in tearDown(). So it's ok to keep some child processes between two unit tests.

I wrote a first change to help debugging the issue. My change adds more warnings and calls support.reap_children():
https://github.com/python/cpython/pull/2841

With this PR, the following test fails:
---
haypo@selma$ ./python -u -m test -v --fail-env-changed --match=test.test_multiprocessing_spawn.WithThreadsTestPool.test_release_task_refs --match=test.test_multiprocessing_spawn.WithProcessesTestLogging.test_level test_multiprocessing_spawn 
== CPython 3.7.0a0 (heads/master:b364d9f, Jul 24 2017, 11:06:33) [GCC 6.3.1 20161221 (Red Hat 6.3.1-1)]
== Linux-4.11.9-200.fc25.x86_64-x86_64-with-fedora-25-Twenty_Five little-endian
== hash algorithm: siphash24 64bit
== cwd: /home/haypo/prog/python/master/build/test_python_24433
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
Run tests sequentially
0:00:00 load avg: 0.27 [1/1] test_multiprocessing_spawn
test_level (test.test_multiprocessing_spawn.WithProcessesTestLogging) ... ok
Warning -- reap_children() reaped child process 24437
test_release_task_refs (test.test_multiprocessing_spawn.WithThreadsTestPool) ... ok
Warning -- reap_children() reaped child process 24438
Dangling processes: {<Process(Process-2, started daemon)>, <Process(Process-1, started daemon)>}

----------------------------------------------------------------------
Ran 2 tests in 1.369s

OK
Warning -- multiprocessing.process._dangling was modified by test_multiprocessing_spawn
  Before: <_weakrefset.WeakSet object at 0x7fdc24f70058>
  After:  <_weakrefset.WeakSet object at 0x7fdc2a666260> 
test_multiprocessing_spawn failed (env changed)

1 test altered the execution environment:
    test_multiprocessing_spawn

Total duration: 2 sec
Tests result: ENV CHANGED
---
History
Date User Action Args
2017-07-24 10:43:31vstinnersetrecipients: + vstinner, pitrou, martin.panter, serhiy.storchaka, davin
2017-07-24 10:43:31vstinnersetmessageid: <1500893011.13.0.321287040227.issue26762@psf.upfronthosting.co.za>
2017-07-24 10:43:31vstinnerlinkissue26762 messages
2017-07-24 10:43:30vstinnercreate