This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Speed difference between Python 2.5 and 2.6 during filling bsddb database.
Type: performance Stage: test needed
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: jafo Nosy List: PeterL, guy.linton, jafo, jcea
Priority: normal Keywords:

Created on 2010-04-24 08:11 by PeterL, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
statistics_for_python_25_26_run.txt.tar.gz PeterL, 2010-04-24 08:11 Profiling results
Messages (5)
msg104067 - (view) Author: Peter Landgren (PeterL) Date: 2010-04-24 08:11
The time it takes, in the application Gramps, to fill an empty bsddb database by importing an XML backup or a GECDOM file, incrises from about 2 minutes to about an hour in Windows XP ana Windows 7. No such degradation has been sen in Linux.

The Gramps code was the same in all test cases.
The running conditions were:
 
Python 2.5                     Python 2.6
Windows  4.4.5.3 (4, 6, 20)    4.7.3 (4.7.25)
Linux    4.4.5.3 (4, 6, 21)    4.7.3 (4.7.25)

Note one little version difference between Windows and Python.

If I install bsddb3 and change Gramps code for that, no noticable speed degradation can be seen.
Windows only with Python 2.6  bsddb3 4.8.4 (4.8.26).

I have run profiling and attach the results.

(Sorry for the fuzz I made in issue 8504.)

The only way of providing a test case,as far as I can find, is to install Gramps, create a new Family Tree (empty database) and import an test XML backup. There are two testcases (*.gramps)  available in:
http://gramps.svn.sourceforge.net/viewvc/gramps/branches/maintenance/gramps32/example/gramps/

Gramps can be found at: 
http://www.gramps-project.org/wiki/index.php?title=Installation
msg104087 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2010-04-24 13:16
Please, install Berkeley DB 4.6.X in linux and reinstall bsddb3 to use it, instead of BDB 4.7. Would be a good idea to install pybsddb 4.7.3, too, to mimic Windows environment under linux closely.

After doing this, and verifying that linux resemble now the Windows environment, try the DB load again.

I need to confirm if the issue is in bsddb module, Berkeley DB code or BDB interaction with Windows OS/Filesystem.

Anyway, if installing a modern bsddb3 (note that we have 5.0.0 available) makes the problem go away, it is a non-issue. Python 2.6 is basically in security/crashes fixes only now. Or very close to it.

With python 2.7 almost ready, I would try the beta too, to confirm this problem is not a problem anymore.
msg104118 - (view) Author: Peter Landgren (PeterL) Date: 2010-04-24 19:12
1. Sorry, I made a mistake this morning. (Had to run to a funeral.)
These are the correct version:
Python 2.5                     Python 2.6
Windows  4.4.5.3 (4, 4, 20)    4.7.3 (4.7.25)
Linux    4.4.5.3 (4, 6, 21)    4.7.3 (4.7.25)

So, the same versions of bsddb and DB in Python 2.6 gives the slow speed performance with Windows but not with Linux. This means that the Windows and Linux environments are equal as far as I can see.
 
2. I installed bsddb3 5.0.0 without any problem, but I had to move libdb48.dll from c:\Python26\bsddb3\utils\
to c:\Python26\Lib\site-packages\bsddb3\
otherwise it could not be found. Any explanation for this?

3. Could not run Gramps in Windows with Py 2.7 as Gramps needs pygtk, pycairo and pygobject to run.

It seems to be a strange issue. It can be worked around by using bsddb3 in stead in Gramps for those who needs it. It is only a problem when you import a backup or a GEDCOM and when you rebuild reference maps, which you don't do very often. It's not a issue with normal usage of Gramps.

So, maybe let it wait until 2.7 is out?
msg104122 - (view) Author: Tim Lyons (guy.linton) Date: 2010-04-24 19:25
I see the same slowdown in Mac OS X with python 2.6 and bsddb 4.7.3 (4.7.25). (So, same version as Windows, and same versions as apparently run OK in linux).

I rely on MacPorts to use these products, so have no idea how I would use bdsdb3 still less python 2.7.
msg130802 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2011-03-14 12:03
Python 2.7 is out for a year now, and BSDDB is not present in Python 3.

I mark this as closed/out of date.

Any bug in pybsddb must be reported to the external package at http://www.jcea.es/programacion/pybsddb.htm
History
Date User Action Args
2022-04-11 14:57:00adminsetgithub: 52762
2011-03-14 12:03:48jceasetstatus: open -> closed

nosy: + jafo
messages: + msg130802

assignee: jcea -> jafo
resolution: out of date
2010-04-24 19:25:30guy.lintonsetmessages: + msg104122
2010-04-24 19:12:26PeterLsetmessages: + msg104118
2010-04-24 13:41:39jceasetnosy: + guy.linton
2010-04-24 13:17:53jceasetassignee: jcea
2010-04-24 13:16:28jceasetmessages: + msg104087
2010-04-24 08:45:08ezio.melottisetpriority: normal
nosy: + jcea

stage: test needed
2010-04-24 08:11:58PeterLcreate