This is a test script, to help me to understand why I have unexpected
result in application.
But I got a more unexpected result, and probably wrong error message
about the read-only cursor.
The full script is at the end.

I got this only one ! I restart the script after the crash, but
this has freezed my xen virtual machine after some minutes. I had to
"destroy" it and restart it. 
After restart was not working anymore, blocked, not responding except to
CTRL-C
I had to drop the old DB, and then i worked again 

I run python 2.5.2 inside a kolab 2.2.0 (aka openpkg) environement 
on a xen centos 5.x running kernel 2.6.18-8.el5xen and glibc-2.5-12

The host is a P4 running centos 5.X and kernel 2.6.18-53.1.4.el5xen


Here are the errors, I waited for long time after the  firts backtrace,
to press CTRL-C and got immediately the second backtrace.

Regards

db cleanup, 326 records deleted, 9990 remains
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/kolab/lib/python/threading.py", line 486, in
__bootstrap_inner
    self.run()
  File "/kolab/lib/python/threading.py", line 446, in run
    self.__target(*self.__args, **self.__kwargs)
  File "./test/test_bdb.py", line 85, in <lambda>
    update_thread=threading.Thread(target=lambda : cleanup(db))
  File "./test/test_bdb.py", line 72, in cleanup
    cursor.delete()
DBPermissionsError: (1, 'Operation not permitted -- Write attempted on
read-
only
cursor')

Traceback (most recent call last):
  File "./test/test_bdb.py", line 130, in <module>
    server()
  File "./test/test_bdb.py", line 92, in server
    db[str(last)]=str(last)
KeyboardInterrupt
*** glibc detected *** /kolab/bin/python: double free or corruption
(fasttop):
0
x09d521a8 ***
======= Backtrace: =========
/lib/i686/nosegneg/libc.so.6[0x4444ccfd]
/lib/i686/nosegneg/libc.so.6(cfree+0x90)[0x444503b0]
/kolab/bin/python(__db_c_destroy+0x60)[0x81fa6f0]
/kolab/bin/python(__db_refresh+0x165)[0x81f3515]
/kolab/bin/python(__db_close+0x64)[0x81f3cb4]
/kolab/bin/python(__db_close_pp+0xa1)[0x8204b61]
/kolab/bin/python[0x812123f]
/kolab/bin/python[0x8159534]
/kolab/bin/python[0x815fb0a]
/kolab/bin/python[0x811abbc]
/kolab/bin/python[0x811abc9]
/kolab/bin/python[0x80b0229]
/kolab/bin/python(PyDict_SetItem+0x6e)[0x80b1c0e]
/kolab/bin/python(PyDict_SetItemString+0x42)[0x80b1ce2]
/kolab/bin/python(PyImport_Cleanup+0xd4)[0x8108294]
/kolab/bin/python(Py_Finalize+0xbf)[0x8113f1f]
/kolab/bin/python(Py_Main+0x468)[0x80845c8]
/kolab/bin/python(main+0x22)[0x8084052]
/lib/i686/nosegneg/libc.so.6(__libc_start_main+0xdc)[0x443fbdec]
/kolab/bin/python[0x8083fa1]
======= Memory map: ========
00110000-001d7000 rwxp 00110000 00:00 0
001d7000-001da000 r-xp 00000000 ca:03 973941     /kolab/lib/python/lib-
dynload/
t
ime.so
001da000-001dc000 rwxp 00002000 ca:03 973941     /kolab/lib/python/lib-
dynload/
t
ime.so
001dc000-001e0000 r-xp 00000000 ca:03 973938     /kolab/lib/python/lib-
dynload/
s
trop.so
001e0000-001e2000 rwxp 00003000 ca:03 973938     /kolab/lib/python/lib-
dynload/
s
trop.so
001e2000-001e3000 r-xp 00000000 ca:03 973911     /kolab/lib/python/lib-
dynload/
_
weakref.so
001e3000-001e4000 rwxp 00000000 ca:03 973911     /kolab/lib/python/lib-
dynload/
_
weakref.so
001e4000-001ea000 rwxs 00000000 ca:03 1747650    /tmp/db/__db.001
001ea000-001fc000 rwxs 00000000 ca:03 1747651    /tmp/db/__db.002
001fc000-0023e000 rwxs 00000000 ca:03 1747652    /tmp/db/__db.003
00388000-003e0000 rwxs 00000000 ca:03 1747653    /tmp/db/__db.004
00450000-00452000 r-xp 00000000 ca:03 1132355    /lib/libutil-2.5.so
00452000-00453000 r-xp 00001000 ca:03 1132355    /lib/libutil-2.5.so
00453000-00454000 rwxp 00002000 ca:03 1132355    /lib/libutil-2.5.so
00459000-0047e000 r-xp 00000000 ca:03 1132349    /lib/i686/nosegneg/
libm-2.5.so
0047e000-0047f000 r-xp 00024000 ca:03 1132349    /lib/i686/nosegneg/
libm-2.5.so
0047f000-00480000 rwxp 00025000 ca:03 1132349    /lib/i686/nosegneg/
libm-2.5.so
0063e000-0063f000 r-xp 0063e000 00:00 0          [vdso]
0097a000-0098d000 r-xp 00000000 ca:03 1134447    /lib/i686/nosegneg/
libpthread-2                                                           
                                                                      .
5.so
0098d000-0098e000 r-xp 00012000 ca:03 1134447    /lib/i686/nosegneg/
libpthread-2                                                           
                                                                      .
5.so
0098e000-0098f000 rwxp 00013000 ca:03 1134447    /lib/i686/nosegneg/
libpthread-2                                                           
                                                                      .
5.so
0098f000-00991000 rwxp 0098f000 00:00 0
009a8000-009e9000 rwxp 009a8000 00:00 0
00b80000-00d80000 r-xp 00000000 ca:03 845325     /usr/lib/locale/
locale-archive
00ec9000-00ecc000 r-xp 00000000 ca:03 973916     /kolab/lib/python/lib-
dynload/
c
StringIO.so
00ecc000-00ecd000 rwxp 00003000 ca:03 973916     /kolab/lib/python/lib-
dynload/
c
StringIO.so
00fd2000-00fd6000 r-xp 00000000 ca:03 973918     /kolab/lib/python/lib-
dynload/
c
ollections.so
00fd6000-00fd7000 rwxp 00004000 ca:03 973918     /kolab/lib/python/lib-
dynload/
c
ollections.so
00fd7000-00fd8000 --xp 00fd7000 00:00 0
00fd8000-019d8000 rwxp 00fd8000 00:00 0
08048000-08433000 r-xp 00000000 ca:03 430731     /kolab/bin/python
08433000-0846e000 rwxp 003eb000 ca:03 430731     /kolab/bin/python
0846e000-08478000 rwxp 0846e000 00:00 0
09ce5000-09da5000 rwxp 09ce5000 00:00 0
443c4000-443dd000 r-xp 00000000 ca:03 1132323    /lib/ld-2.5.so
443dd000-443de000 r-xp 00018000 ca:03 1132323    /lib/ld-2.5.so
443de000-443df000 rwxp 00019000 ca:03 1132323    /lib/ld-2.5.so
443e6000-44521000 r-xp 00000000 ca:03 1134399    /lib/i686/nosegneg/
libc-2.5.so
44521000-44523000 r-xp 0013a000 ca:03 1134399    /lib/i686/nosegneg/
libc-2.5.so
44523000-44524000 rwxp 0013c000 ca:03 1134399    /lib/i686/nosegneg/
libc-2.5.so
44524000-44527000 rwxp 44524000 00:00 0
44529000-4452b000 r-xp 00000000 ca:03 1134402    /lib/libdl-2.5.so
4452b000-4452c000 r-xp 00001000 ca:03 1134402    /lib/libdl-2.5.so
4452c000-4452d000 rwxp 00002000 ca:03 1134402    /lib/libdl-2.5.so
44622000-4462d000 r-xp 00000000 ca:03 1134413    /lib/
libgcc_s-4.1.1-20070105.so                                             
                                                                       
            .
1Aborted

#!/kolab/bin/python

import os, sys, logging, time, threading
import marshal
import bsddb.db

# this is a test to see how bsddb is reliable
# The server count in the db, and the client search for missing value
# run on server in a terminal, and a client in another

#
======================================================================
# db functions
#
======================================================================

db_dir='/tmp/db'
db_filename=os.path.join(db_dir, 'test.bdb')

try:
    os.mkdir(db_dir)
except OSError:
    pass

freq=100 # new sample per/sec
timeout=3 # in sec, how long I wait for them
retention=100 # in sec, how long they stay in the db

def init_db(read_only=True):
    # open the environement
    dbenv=db=None
    try:
        db_env=bsddb.db.DBEnv(0)

        flags=bsddb.db.DB_CREATE | bsddb.db.DB_INIT_CDB |
bsddb.db.DB_INIT_MPOOL
        db_env.open(db_dir, flags)
        db=bsddb.db.DB(db_env)
        flags = bsddb.db.DB_DIRTY_READ | ( bsddb.db.DB_RDONLY if
read_only else bsddb.db.DB_CREATE )

        db.open(db_filename, dbtype=bsddb.db.DB_HASH,flags=flags)
    except Exception:
        print 'error opening db'
        raise
    else:
        print 'db opened'

    return dbenv, db

#
----------------------------------------------------------------------
def close_db(dbenv, db):
    try:
        if db:
            db.close()
        if dbenv:
            db_env.close()
    except Exception:
        log.rcpt.exception('closing db')
    else:
        log.rcpt.info('db closed')

#
----------------------------------------------------------------------
def now():
    return int(time.time()*freq)

def cleanup(db):

    while True:
        cursor=db.cursor(None, bsddb.db.DB_WRITECURSOR)
        count=remain=0
        c=cursor.first()
        limit=now()-retention*freq
        while c:
            k, v=c
            t=int(k)
            if t<=limit:
                cursor.delete()
                count+=1
            else:
                remain+=1
            c=cursor.next()
        cursor.close()
        print 'db cleanup, %d records deleted, %d remains' % (count,
remain)
        time.sleep(timeout)

def server():
    dbenv, db=init_db(read_only=False)

    update_thread=threading.Thread(target=lambda : cleanup(db))
    update_thread.setDaemon(True)
    update_thread.start()
    last=now()
    while True:

        while last<now():
            db[str(last)]=str(last)
            last+=1

        time.sleep(1/freq)

    close_db(dbenv, db)

def client():

    dbenv, db=init_db(read_only=True)

    max_delay=0
    last=now()

    while True:
        try:
            value=db[str(last)]
        except KeyError:
            if last<now()-timeout*freq:
                print 'not found', last, float(now()-last)/freq
                last+=1
            else:
              time.sleep(1/freq)
        else:
            delay=now()-last
            if delay>max_delay:
                max_delay=delay
                print 'delay', max_delay

            last+=1

    close_db(dbenv, db)

if sys.argv[1]=='c':
    client()
elif sys.argv[1]=='s':
    server()

Alain, you are seeing a resource exhaustion problem. Somewhere memory is
leaking. That is, the programs are leaking memory.

bsddb was vastly improved in python 2.6. In particular, memory leaks and
resource tree deallocations. Can you consider use that version?.

You can install and updated pybsddb version in python 2.5. See
http://www.jcea.es/programacion/pybsddb.htm. Just remember that the
module is called "bsddb3" instead of "bsddb". You can easily install it
using setuptools ("easy_install" script).

Please, confirm that the problem vanished with the new module.

What Berkeley DB are you using with bsddb bindings?.

Alain, I can not reproduce this issue in current bsddb code (as running
under Python 2.6.1 and Berkeley DB 4.7.25). Since Python 2.5 branch has
been marked as "security fixes only", this bug would be an "out of date"
one.

Please, confirm that this bug is not present in Python 2.6.1 or, if you
need to run Python 2.5, please install an updated pybsddb library and
check it.

I will wait your comments for a week or two. Then mark this issue as
"out of date", if you don't indicate the contrary.

Thanks for your time.

Closing the issue.

Python 2.5.x is only open for security patches. bsddb library in Python
2.6 doesn't have this issue.

People needing recent pybsddb in old python releases can download from
http://www.jcea.es/programacion/pybsddb.htm