classification
Title: bsddb3 intermittent test failures
Type: behavior Stage: needs patch
Components: Tests Versions: Python 2.7, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: jcea Nosy List: BreamoreBoy, haypo, jcea, r.david.murray
Priority: normal Keywords: buildbot

Created on 2009-07-11 16:32 by r.david.murray, last changed 2014-06-20 22:50 by BreamoreBoy.

Messages (7)
msg90426 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-11 16:32
The buildbots periodically report failures in the bsddb3 test suite. 
These are not reproducible and presumably are timing-based.  It would be
best if the tests could be made to pass consistently, to make the
buildbot output more useful.

Here is one such failure that I see on a semi-regular basis on trunk
(this happens in my own tests runs as well, also not consistently):

test_bsddb3
Berkeley DB 4.7.25: (June 30, 2009)
Test path prefix:  /tmp/z-test_bsddb3-11362
test test_bsddb3 failed -- Traceback (most recent call last):
  File "/home/rdmurray/python/trunk/Lib/bsddb/test/test_replication.py",
line 134, in test01_basic_replication
    self.assertTrue(time.time()<timeout)
AssertionError: False is not True

I will add others here as I capture examples.
msg90732 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-07-20 14:47
Another buildbot intermittent failure, this one from

  http://www.python.org/dev/buildbot/all/x86%20XP-4%20trunk/builds/2332

ie: the x86 XP-4 buildbot:

Traceback (most recent call last):
  File
"E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\threading.py",
line 524, in __bootstrap_inner
    self.run()
  File
"E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\threading.py",
line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File
"E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\bsddb\test\test_thread.py",
line 306, in readerThread
    rec = dbutils.DeadlockWrap(c.next, max_retries=10)
  File
"E:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\bsddb\dbutils.py",
line 68, in DeadlockWrap
    return function(*_args, **_kwargs)
DBLockDeadlockError: (-30994, 'DB_LOCK_DEADLOCK: Locker killed to
resolve a deadlock')

That run also shows the replication timeout error, which that buildbot
often produces.
msg94861 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-11-03 14:16
Unless someone objects, I'm going to disable the replication timeout
check unconditionally, since it is the worst culprit for buildbot
stability on 2.x.  Note that issue 3892 indicates it happens
consistently on windows, and there was already a partial disable for
that platform.  All the other tests usually succeed even if that
particular assert fails.
msg94915 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-11-05 01:54
I was able to consistently reproduce the timeout on my Gentoo buildbot
running in a vserver virthost.  After some experimentation and perusing
the related issue 3892, I discovered that by increasing the timeout I
could get the test to consistently pass in my buildbot.  However, it
does occasionally produce the DB_LOCK_DEADLOCK traceback in one of the
threads. (Two times out of about 17 runs in a loop).

I've checked an increase in the timeout for the two replication tests
into trunk in r76116.  In some of my experiments it was clear that the
base replication case is much quicker than the ReplicationManager case,
so I made the ReplicationManager timeout longer.  I'm hoping this will
fix the line 315 failure in the windows buildbot, since it does
sometimes pass.
msg94959 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-11-06 02:40
The deadlock error is still there, so I'm leaving this open, but it is
no longer causing buildbot instability.
msg104188 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2010-04-26 00:04
Is the following error related to this issue?

http://www.python.org/dev/buildbot/builders/x86 Windows7 2.6/builds/256/steps/test/logs/stdio

test test_bsddb3 failed -- Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\2.6.bolen-windows7\build\lib\bsddb\test\test_lock.py", line 127, in test03_lock_timeout
    self.assertTrue((end_time-start_time) >= 0.0999)
AssertionError
msg221129 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-20 22:50
@David this is now out of date?
History
Date User Action Args
2014-06-20 22:50:42BreamoreBoysetnosy: + BreamoreBoy
messages: + msg221129
2010-08-22 20:28:06floxsetkeywords: + buildbot
2010-04-26 00:04:33hayposetnosy: + haypo
messages: + msg104188
2009-11-06 02:41:08r.david.murraysetkeywords: - buildbot
2009-11-06 02:40:21r.david.murraysetmessages: + msg94959
2009-11-05 01:54:03r.david.murraysetmessages: + msg94915
2009-11-03 14:16:21r.david.murraysetmessages: + msg94861
versions: + Python 2.6
2009-10-30 19:46:37r.david.murraysetkeywords: + buildbot
2009-07-20 14:47:32r.david.murraysetmessages: + msg90732
2009-07-11 16:32:11r.david.murraycreate