classification
Title: transient failure in test_multiprocessing.WithProcessesTestCondition
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.2, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: neologix, pitrou, python-dev, vstinner
Priority: normal Keywords:

Created on 2011-04-06 23:04 by pitrou, last changed 2017-05-15 15:32 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 1591 merged vstinner, 2017-05-15 14:26
Messages (6)
msg133182 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-06 23:04
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%202.7/builds/519/steps/test/logs/stdio

======================================================================
FAIL: test_notify_all (test.test_multiprocessing.WithProcessesTestCondition)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/db3l/buildarea/2.7.bolen-freebsd7/build/Lib/test/test_multiprocessing.py", line 757, in test_notify_all
    self.assertReturnsIfImplemented(6, get_value, woken)
  File "/usr/home/db3l/buildarea/2.7.bolen-freebsd7/build/Lib/test/test_multiprocessing.py", line 116, in assertReturnsIfImplemented
    return self.assertEqual(value, res)
AssertionError: 6 != 5
msg133533 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-04-11 16:08
One possible cause for those intermittent failures is the preemtion of a thread while waiting on the condition:

 def wait(self, timeout=None):
   233         assert self._lock._semlock._is_mine(), \
   234                'must acquire() condition before using wait()'
   235 
   236         # indicate that this thread is going to sleep
   237         self._sleeping_count.release()
   238 
   239         # release lock
   240         count = self._lock._semlock._count()
   241         for i in range(count):
   242             self._lock.release()
   243 
<-- here
   244         try:
   245             # wait for notification or timeout
   246             ret = self._wait_semaphore.acquire(True, timeout)


For example, if the last thread/process is preempted after having released the condition's lock (and hence performed a up on the "sleeping" semaphore sooner in the "f" function) but before waiting on the condition's semaphore, since the main thread only waits 0.1s before locking the condition and performing a notify_all on it (it will proceed since all the threads performed an up on "sleeping"), only the threads already waiting on the condition will be woken up, this last thread won't be woken up, triggering a failure in this assertion
   764         self.assertReturnsIfImplemented(0, get_value, woken)
with woken.get_value() == 5

It's just a guess, but I'd suggest increasing the sleep before trying to signal the condition a bit:

   762         # check no process/thread has woken up
   763         time.sleep(10 * DELTA)
msg133536 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011-04-11 16:18
Sorry, wrong copy-paste, the failing assertion will of course be this one:
   773         self.assertReturnsIfImplemented(6, get_value, woken)

since woken.get_value() == 5
msg133901 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-04-16 18:59
Indeed, it just seems that the sleep period is sometimes too low. Will commit a patch.
msg133902 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-04-16 19:03
New changeset 88f1907fe312 by Antoine Pitrou in branch '3.2':
Issue #11790: Fix sporadic failures in test_multiprocessing.WithProcessesTestCondition.
http://hg.python.org/cpython/rev/88f1907fe312

New changeset 0ecfa2ce6561 by Antoine Pitrou in branch 'default':
Issue #11790: Fix sporadic failures in test_multiprocessing.WithProcessesTestCondition.
http://hg.python.org/cpython/rev/0ecfa2ce6561
msg293714 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-05-15 15:32
New changeset 9d1983be503012e750f49d31b569f3fe4554c628 by Victor Stinner in branch '2.7':
bpo-11790: Fix sporadic failures in test_multiprocessing.WithProcessesTestCondition. (#1591)
https://github.com/python/cpython/commit/9d1983be503012e750f49d31b569f3fe4554c628
History
Date User Action Args
2017-05-15 15:32:16vstinnersetnosy: + vstinner
messages: + msg293714
2017-05-15 14:26:30vstinnersetpull_requests: + pull_request1684
2011-04-16 19:03:59pitrousetstatus: open -> closed
stage: resolved
resolution: fixed
versions: - Python 3.1, Python 2.7
2011-04-16 19:03:07python-devsetnosy: + python-dev
messages: + msg133902
2011-04-16 18:59:36pitrousetmessages: + msg133901
2011-04-11 16:18:22neologixsetmessages: + msg133536
2011-04-11 16:08:29neologixsetnosy: + neologix
messages: + msg133533
2011-04-06 23:04:34pitroucreate