New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_multiprocessing_spawn fails when ran with -Werror #65978
Comments
$ ./python -Werror -m test.regrtest -v -m test_sys_exit test_multiprocessing_spawn
== CPython 3.5.0a0 (default:149cc6364180+, Jun 12 2014, 15:45:54) [GCC 4.6.3]
== Linux-3.8.0-36-generic-i686-with-debian-wheezy-sid little-endian
== hash algorithm: siphash24 32bit
== /home/serhiy/py/cpython/build/test_python_9425
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_multiprocessing_spawn
test_sys_exit (test.test_multiprocessing_spawn.WithProcessesTestSubclassingProcess) ... Exception ignored in: <_io.FileIO name='@test_9425_tmp' mode='wb'>
ResourceWarning: unclosed file <_io.TextIOWrapper name='@test_9425_tmp' mode='w' encoding='UTF-8'>
FAIL ====================================================================== Traceback (most recent call last):
File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 483, in test_sys_exit
self.assertEqual(f.read().rstrip(), str(reason))
AssertionError: "[1, 2, 3]\nException ignored in: <_io.Fi[123 chars]-8'>" != '[1, 2, 3]'
- [1, 2, 3]
? -
+ [1, 2, 3]- Exception ignored in: <_io.FileIO name='/dev/null' mode='rb'>
- ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'> Ran 1 test in 1.247s FAILED (failures=1) |
This can be reproduced on Windows 8.1. |
Attaching patch for both 3.4 and default/3.5 (single file works for both) which fixes the test to properly adjust for running under -Werror. Whether -Werror is set or otherwise the equivalent of 'warnings.simplefilter("error", Warning)' has been set, any warnings will trigger messages being sent to stderr (that's good). This problematic test, in particular, reassigns sys.stderr to point at a file in an attempt to capture exit codes from a Process into that file (that's okay); but any warning messages sent to stderr may also end up being captured inside that same file depending upon the timings of the threads involved (that's bad). To support the established behavior inside multiprocessing for capturing non-int exit codes (see bpo-13854), the test has been made more robust to detect when it is being run with -Werror (or similar) and allow for extra messages potentially appearing on stderr, otherwise it will continue to use the existing test to validate the output seen on stderr (captured in that file). In the suite of tests for multiprocessing, test_multiprocessing_spawn currently triggers a bunch of warnings due to differences in how spawn works versus fork -- one example of this common pattern:
How to better handle the triggering of these warnings in the first place is left as a topic for consideration in the future. This patch's updated test has been tested with and without "-Werror" against 3.4 and default/3.5 on OS X 10.10. |
Wouldn't the test fail with -Werror::ResourceWarning or like? |
Yes, setting -Werror::ResourceWarning would indeed cause the test to fail again. A couple of options we could potentially adopt:
Of those options, I do not like #1 and I think #3 is a slippery slope to step onto -- this patch pragmatically offers us #2 with the potential to pursue #4 (which is ultimately where I think we want this to go). I kind of like the thought of someone specifying -Werror::ResourceWarning and getting to see this test fail. |
Proposed patch cures only the symptom, the failure of one rigid test. A warning still is emitted in applications that use multiprocessing. Closed in favor of duplicate bpo-25654 with patches that are intended to fix the cause of warnings. In any case thank you for your contribution Davin. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: