classification
Title: intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.4, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: berker.peksag, davin, jnoller, python-dev, sbt, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-03-19 20:05 by davin, last changed 2015-04-23 08:37 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
issue_23713_fix_fragile_imap_unordered_test_py35_and_py34.patch davin, 2015-03-19 20:08 Patch for default/3.5 and 3.4 (both). review
issue_23713_fix_fragile_imap_unordered_test_py27.patch davin, 2015-03-19 20:41 Patch for 2.7 branch. review
Messages (9)
msg238553 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-03-19 20:05
test_imap_unordered_handle_iterable_exception fails intermittently on some platforms due to the test being too rigid (fragile) about the order of the items being iterated over which, as the name would imply, is not guaranteed.

Of the multiprocessing module's test modules, test_multiprocessing_fork, test_multiprocessing_forkserver, and test_multiprocessing_spawn all leverage the unit test test_imap_unordered_handle_iterable_exception in multiple different contexts.  There are thus multiple opportunities for test_imap_unordered_handle_iterable_exception to fail due to this fragility -- it's maybe a little bit surprising the test doesn't fail more often than it does.

A patch is forthcoming.
msg238554 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-03-19 20:08
Attaching patch for this fragile test for default/3.5 and 3.4 branches.

Blame for implementing the fragile test falls to @davin.  In my own defense, @davin also discovered it in testing on FreeBSD 10.1 i386-RELEASE.
msg238555 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-03-19 20:41
Attaching patch for this fragile test for 2.7 branch.

Patches have been put through test.regrtest on OS X 10.10 (64-bit), FreeBSD 10.1 i386 (32-bit), and Raspbian Debian (ARMv6) thus far.
msg239481 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-29 08:08
Why not use just

    self.assertEqual(sorted(it), list(map(sqr, list(range(10)))))

or

    self.assertCountEqual(list(it), list(map(sqr, list(range(10)))))

?
msg240285 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-04-08 20:39
Serhiy: If I understand correctly what you suggest, calling sorted(it) or list(it) would run the iterator all the way until it raises the SayWhenError exception, triggering the self.assertRaises before it ever actually gets to call the self.assertEqual or self.assertCountEqual you suggest.  It would indeed test the raising of the exception but would never test individual values returned by the iterator to see that they belong.

Or were you suggesting something different?
msg241801 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2015-04-22 13:36
Just saw this on x86 Tiger 3.x:

======================================================================
FAIL: test_imap_unordered_handle_iterable_exception (test.test_multiprocessing_fork.WithProcessesTestPool)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/_test_multiprocessing.py", line 1805, in test_imap_unordered_handle_iterable_exception
    self.assertEqual(next(it), i*i)
AssertionError: 1 != 0

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/9559/steps/test/logs/stdio

The patch looks good to me. I'm not familiar with the test code, but Serhiy's suggestion also looks good to me.
msg241826 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015-04-22 21:16
Serhiy:  If my comments on your questions make sense, should we go ahead with the patch as it is?
msg241851 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-23 08:31
Thanks for your explanation and sorry for the delay Davin. Yes, it makes sense.
msg241852 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-04-23 08:37
New changeset 0ac30526c208 by Serhiy Storchaka in branch '2.7':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0ac30526c208

New changeset 0eb5968c15ad by Serhiy Storchaka in branch '3.4':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0eb5968c15ad

New changeset f60f65507d8e by Serhiy Storchaka in branch 'default':
Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/f60f65507d8e
History
Date User Action Args
2015-04-23 08:37:49serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2015-04-23 08:37:23python-devsetnosy: + python-dev
messages: + msg241852
2015-04-23 08:31:39serhiy.storchakasetassignee: davin -> serhiy.storchaka
messages: + msg241851
stage: patch review -> commit review
2015-04-22 21:16:30davinsetmessages: + msg241826
2015-04-22 13:36:38berker.peksagsetnosy: + berker.peksag
messages: + msg241801
2015-04-08 20:39:36davinsetmessages: + msg240285
2015-03-29 08:08:09serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg239481
2015-03-19 20:41:23davinsetfiles: + issue_23713_fix_fragile_imap_unordered_test_py27.patch

messages: + msg238555
stage: patch review
2015-03-19 20:08:47davinsetfiles: + issue_23713_fix_fragile_imap_unordered_test_py35_and_py34.patch
keywords: + patch
messages: + msg238554
2015-03-19 20:05:03davincreate