Deprecate bsddb for removal in 3.0 #48019

brettcannon · 2008-09-03T23:06:32Z

BPO	3769
Nosy	@smontanaro, @warsaw, @brettcannon, @rhettinger, @jcea, @ncoghlan, @benjaminp
Superseder	bpo-3776: deprecate bsddb/dbhash in 2.6 for removal in 3.0
Files	deprecate_bsddb.diff: Deprecate bsddb and dbhash

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2008-09-04.01:27:32.933>
created_at = <Date 2008-09-03.23:06:31.831>
labels = ['deferred-blocker', 'type-bug', 'library']
title = 'Deprecate bsddb for removal in 3.0'
updated_at = <Date 2008-09-05.18:19:35.654>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2008-09-05.18:19:35.654>
actor = 'brett.cannon'
assignee = 'none'
closed = True
closed_date = <Date 2008-09-04.01:27:32.933>
closer = 'barry'
components = ['Library (Lib)']
creation = <Date 2008-09-03.23:06:31.831>
creator = 'brett.cannon'
dependencies = []
files = ['11367']
hgrepos = []
issue_num = 3769
keywords = ['patch', 'needs review']
message_count = 19.0
messages = ['72431', '72432', '72433', '72434', '72435', '72436', '72437', '72438', '72441', '72444', '72449', '72468', '72476', '72503', '72506', '72511', '72557', '72597', '72615']
nosy_count = 7.0
nosy_names = ['skip.montanaro', 'barry', 'brett.cannon', 'rhettinger', 'jcea', 'ncoghlan', 'benjamin.peterson']
pr_nums = []
priority = 'deferred blocker'
resolution = 'duplicate'
stage = None
status = 'closed'
superseder = '3776'
type = 'behavior'
url = 'https://bugs.python.org/issue3769'
versions = ['Python 3.0']

brettcannon · 2008-09-03T23:06:31Z

Attached is a patch that deprecates bsddb for removal in 3.0.

benjaminp · 2008-09-03T23:12:56Z

I also think you need to deprecate the dbhash module.

smontanaro · 2008-09-03T23:20:54Z

Remind me why we want to get rid of bsddb?

Skip

benjaminp · 2008-09-03T23:22:38Z

On Wed, Sep 3, 2008 at 6:20 PM, Skip Montanaro <report@bugs.python.org> wrote:

Skip Montanaro <skip@pobox.com> added the comment:

Remind me why we want to get rid of bsddb?

The reasons are enumerated in PEP-3108.

Skip

----------
nosy: +skip.montanaro

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue3769\>

--
Cheers,
Benjamin Peterson
"There's no place like 127.0.0.1."

rhettinger · 2008-09-03T23:28:01Z

I thought someone stepped forward to maintain this package.

benjaminp · 2008-09-03T23:32:32Z

Also see this:
http://mail.python.org/pipermail/python-3000/2008-September/014712.html

brettcannon · 2008-09-03T23:40:16Z

New patch to also deprecated dbhash.

rhettinger · 2008-09-03T23:54:29Z

I think this should be deferred to Py3.1.

This decision was not widely discussed and I think it likely that some
users will be surprised and dismayed. The release candidate seems to be
the wrong time to yank this out (in part because of the surprise factor)
and in part because I think the change silently affects shelve/anydbm
performance so that the impact may be significantly degraded but not
immediately apparent (test suites will still succeed).

We don't have to take this out. So why do it hastily at the last minute
and without some discussion on comp.lang.python at least.

If it were any other release, we would have disciplined ourselves to
deprecate first and remove a generation or two later.

Also, the stated reason for removal may yet disappear if jcrea steps in
as promised and continues to make updates.

Also, the referenced note (
http://mail.python.org/pipermail/python-dev/2008-July/081379.html ) say
to "start end-of-lifing it" which I took to mean deprecate rather than
completely remove during a release candidate.

brettcannon · 2008-09-04T00:12:42Z

On Wed, Sep 3, 2008 at 4:41 PM, Raymond Hettinger <python@rcn.com> wrote:

I think this should be deferred to Py3.1.
This decision was not widely discussed and I think it likely that some users
will
be surprised and dismayed.

Perhaps, but that could be said about almost any module that has been
removed through the stdlib reorg.

The release
candidate seems to be the wrong time to
yank this out (in part because of the surprise
factor) and in part because I think the change
silently affects shelve performance so that the
impact may be significantly negative but not
readily apparent.

We don't have to take this out.

We don't have to remove anything that has gone through the stdlib
reorg, so that is not a solid argument.

So why do
it hastily at the last minute and without some
discussion on comp.lang.python at least.

It isn't being done "hastily"; this has been planned for a while.
People have just been too busy to get around to it. And we are not
changing any semantics or removing something from the language which
is what I view as what you don't change in an rc. So this might come
down to a different opinion of what one can do during an rc.

If it were any other release, we would have
disciplined ourselves to deprecate first and
remove a generation or two later.

We are deprecating first in 2.6.

Also, the reason for removal may yet disappear
if jcrea steps in an continues to make updates.

OK, but none of his changes have received a code review, so if we are
going to go down the whole "disciplined" route about it being an rc
then we should back out all of Jesus' changes for both 2.6 and 3.0,
which puts us back to the same instability issues.

Also, the referenced note (
http://mail.python.org/pipermail/python-dev/2008-July/081379.html )
say to "start end-of-lifing it" which I took to mean deprecate rather than
remove during a release candidate.

Well, it was in the PEP before beta2 even went out the door.

-Brett

rhettinger · 2008-09-04T01:09:16Z

> I think this should be deferred to Py3.1.
> This decision was not widely discussed and I think
> it likely that some users will be surprised and dismayed.

Perhaps, but that could be said about almost any module
that has been removed through the stdlib reorg.

Many of those were trivial in comparison and many had good replacements
or were no longer relevant. This is a bigger deal (almost on par with
removing Tkinter).

> If it were any other release, we would have
> disciplined ourselves to deprecate first and
> remove a generation or two later.

We are deprecating first in 2.6.

That, of course, isn't fair since 2.6 is going out the door at the same
time as 3.0. Normally, we would go through more that one cycle between
the heads-up and taking the hit. Life is already going to be
challenging for people using 2.6 as a stepping stone to 3.0. Would hate
to stop some of them dead in the water.

Well, it was in the PEP before beta2 even went out the door.

If you think everyone got fair warning, knows this is coming, has had a
chance to assess its impact, and has had a chance to comment on it, then
you're kidding yourself.

I don't understand the rush to yank out a major component affecting a
commonly used module (shelve) without more due process, due diligence,
and a bit higher standard of care. I don't think we're respecting our
users on this one. What's wrong with deferring this to 3.1?

warsaw · 2008-09-04T01:27:33Z

See http://mail.python.org/pipermail/python-dev/2008-July/081362.html

Guido states his opinion in no uncertain terms regarding pybsddb in
Python 3.0:

"+1. In my recollection maintaining bsddb has been nothing but trouble
right from the start when we were all sitting together at "Zope Corp
North" in a rented office in McLean... We can remove it from 3.0. We
can't really remove it from 2.6, but we can certainly start
end-of-lifing it in 2.6."

smontanaro · 2008-09-04T02:43:45Z

Is there going to be a dbm.* module which is supported across all the core
platforms: Windows, Mac & Unix? I don't count dumbdbm as really all that
useful, especially given the caveats listed in the module docstring.

If not, could a dbm.sqlite module be written for 2.7 and 3.1 which can fill
that role?

Skip

rhettinger · 2008-09-04T03:22:49Z

Since SQLite has a blob type and allows text keys, we should be able to
make a substitute that doesn't depend on bsddb.

Against, recommend holding-off on removal until 3.1 so we can bake in a
reasonable substitute (esp. for shelves where dumbdbm is a really bad
fallback).

jcea · 2008-09-04T13:36:02Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brett Cannon wrote:

> Also, the reason for removal may yet disappear
> if jcrea steps in an continues to make updates.

OK, but none of his changes have received a code review, so if we are
going to go down the whole "disciplined" route about it being an rc
then we should back out all of Jesus' changes for both 2.6 and 3.0,
which puts us back to the same instability issues.

I was wondering if somebody could write a "TO DO" list of things need to
keep bsddb in the stdlib. Then I could work to trying to comply :).

Yes, we are all very busy guys, but still...

Jesus Cea Avion _// _/// _///
jcea@jcea.es - http://www.jcea.es/ _// _// _// _// _//
jabber / xmpp:jcea@jabber.org _// _// _/////
. _// _// _// _// _//
"Things are not so easy" _// _// _// _// _// _//
"My name is Dump, Core Dump" _/// _/// _// _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSL/kPZlgi5GaxT1NAQLu4AP/VSHPYOCQgQYFJsdi2MWXBpyY7TyC5XgT
Ks2uilXru/hsKQcegn8G6z/53Bt0Uu+oXZSQaZ51V8VnwDXEqaZ+GnKK+S2ky9m0
yomgMlKIZZJsOVd6X4HbLtrVYVKX8wQ224X/yCkw27OLzLIE9IDbUzEjC3+/A7mD
9IJu3B6IaLA=
=ZA8p
-----END PGP SIGNATURE-----

ncoghlan · 2008-09-04T13:49:25Z

Raymond Hettinger wrote:

I think this should be deferred to Py3.1.
This decision was not widely discussed and I think it likely that some
users will
be surprised and dismayed. The release
candidate seems to be the wrong time to
yank this out (in part because of the surprise
factor) and in part because I think the change
silently affects shelve performance so that the
impact may be significantly negative but not
readily apparent.

I don't use Python for database work myself, but something I am somewhat
disappointed to lose is the presence of a moderately complicated package
within the Python distribution itself which is making use of the various
2-to-3 migration tools to support both Python 2.x and 3.x with a single
mixed Python-and-C code base (along with automatic conversion via 2to3).

While that will still be visible to some degree due to the presence of
the 2.x version of the bsddb code in Python 2.6, I don't think it will
be quite the same as it would have been with the 3.x version also being
readily available as part of the standard 3.0 install.

Regardless, given that the removal of bsddb from the 3.0 branch is now a
done deal in svn, I suggest that all we can do is monitor how much
feedback we get indicating that the need to download bsddb as a separate
install is a significant hindrance to Py3k migrations. If the feedback
indicates it is necessary, restoring the module for 3.1 certainly isn't
an impossibility, but in the meantime there *are* real benefits to
decoupling the maintenance cycles (those wanting to get the most out of
Jesus's ongoing work in exposing more of the bsddb API are probably
going to want to be using the external pybsddb releases anyway rather
than waiting however many months it ends up being until we get to 2.7/3.1).

There's also a bit of a second shot at this for bsddb supporters, in
that some of the "omnibus" Python distribution providers like
ActiveState and Enthought may choose to include pybsddb once they start
releasing bundles for Python 3.0.

As far as the corporate scenarios go: if a corporate environment is *so*
screwed up as to simultaneously permit a migration from Python 2.x to
3.0 of an internal project that relies on bsddb, while at the same time
banning those performing the migration from installing the 3.0
compatible version of pybsddb, then they have much bigger problems than
which modules and packages we are choosing to include in the standard
library. In my experience, restrictive corporate environments are far
more likely to just disallow migrations to 3.0 altogether (and in many
cases, the decision makers disallowing such migrations probably won't be
wrong - the case for migrating an existing internal app to 3.0 is far,
far weaker than that for using 3.0 for new development).

Cheers,
Nick.

jcea · 2008-09-04T15:54:16Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:

While that will still be visible to some degree due to the presence of
the 2.x version of the bsddb code in Python 2.6, I don't think it will
be quite the same as it would have been with the 3.x version also being
readily available as part of the standard 3.0 install.

Since 2.6 intention seems to mark this module as deprecated, I guess 2.x
bsddb presence in stock python will finish in 2.7. Moreover, I'm working
just now improving 2.x/3.x conversion code in pybsddb. I think this code
will be available in bsddb 4.7.4, and it will not be integrated in
Python 2.6 (that will include 4.7.3.minor releases, if we keep the
criteria of "only stability and security fixes in 2.6.x").

If the idea is to keep bsddb alive in 2.x, I don't see the point of not
keeping the 3.0 version, because the issues used to justify the removal
persist: I'm the only maintainer, little code review, buildbot issues, etc.

(I would like a comprehensive list, to be able to improve those
deficiencies).

In fact, if we keep bsddb in 2.x, the pressure to keep it in 3.x will be
higher.

Regardless, given that the removal of bsddb from the 3.0 branch is now a
done deal in svn, I suggest that all we can do is monitor how much

Any version control system can revert that with a single command :).

All I can say is that current bsddb code (in my personal repository)
passes all tests in current compiled python3.0 binary, called with the
"-bb" parameter flag (the "-bb" flag was something I learned yesterday).

but in the meantime there *are* real benefits to
decoupling the maintenance cycles (those wanting to get the most out of
Jesus's ongoing work in exposing more of the bsddb API are probably
going to want to be using the external pybsddb releases anyway rather
than waiting however many months it ends up being until we get to 2.7/3.1).

The cycles are actually decoupled since I toke over the bsddb
maintenance (I've released a new version per month). So the release
cycles are not an issue.

The main issue here is 3.0 support, that I worked over the last couple
of months. It is done now. It couldn't be done faster because I was
learning 3.0 internals on-the-fly (there are NO docs about C module
migration; my experience there could be valuable) and 3.0 was a moving
target (still is). For example, when I left to summer holiday bsddb
worked flawless in Python 3.0b2. It failed in 3.0b3 because threading
renames done in python 3.0.

So, Python 3.0 is not waiting for bsddb to be ready, because it already
is (since yesterday). And future Python releases won't suffer because we
won't have any other major architectural reengineering of Python in a
long long time (I hope!).

That is, future Python releases would take whatever bsddb is available
at that time. No wait. No dependent release cycles. With my current
release schema of "a release per month", I can track python evolution
with little effort. For example, Python 2.5 to 2.6 was pretty painless,
even with the "PyBytes_*" ugliness.

As far as the corporate scenarios go: if a corporate environment is *so*
screwed up as to simultaneously permit a migration from Python 2.x to
3.0 of an internal project that relies on bsddb, while at the same time
banning those performing the migration from installing the 3.0
compatible version of pybsddb, then they have much bigger problems than
which modules and packages we are choosing to include in the standard
library.

Agreed. I was thinking about bsddb removal in 2.7.

Jesus Cea Avion _// _/// _///
jcea@jcea.es - http://www.jcea.es/ _// _// _// _// _//
jabber / xmpp:jcea@jabber.org _// _// _/////
. _// _// _// _// _//
"Things are not so easy" _// _// _// _// _// _//
"My name is Dump, Core Dump" _/// _/// _// _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSMAEnplgi5GaxT1NAQIrKgP/YAp45HUSG8Q+M355LTVqlcLMLkycpooc
fflW0MlQ3zZV307VBUbGo9urkS6h1pYhYByivApylhVqj8D4x8OEmMZk0lX8cegG
LYSBzs/sBeyxWWva6r5D9/4DsgJe9ZHqaBLMpy6ipPNVtUbMS61VTNovb3wP+f72
EnSIf9k/glM=
=QxRo
-----END PGP SIGNATURE-----

smontanaro · 2008-09-05T00:01:37Z

me> If not, could a dbm.sqlite module be written for 2.7 and 3.1 which
me> can fill that role?

http://bugs.python.org/issue3783

S

jcea · 2008-09-05T14:51:58Z

This issue is closed but "rejected". Should it be marked as "accepted"
or "fixed"?.

brettcannon · 2008-09-05T18:19:35Z

I accidentally filed this twice, and the code review is on the other
issue, so setting the superceder and closing (although the patch has not
been applied yet).

brettcannon added release-blocker stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Sep 3, 2008

benjaminp added deferred-blocker and removed release-blocker labels Sep 4, 2008

warsaw closed this as completed Sep 4, 2008

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate bsddb for removal in 3.0 #48019

Deprecate bsddb for removal in 3.0 #48019

brettcannon commented Sep 3, 2008

brettcannon commented Sep 3, 2008

benjaminp commented Sep 3, 2008

smontanaro commented Sep 3, 2008

benjaminp commented Sep 3, 2008

rhettinger commented Sep 3, 2008

benjaminp commented Sep 3, 2008

brettcannon commented Sep 3, 2008

rhettinger commented Sep 3, 2008

brettcannon commented Sep 4, 2008

rhettinger commented Sep 4, 2008

warsaw commented Sep 4, 2008

smontanaro commented Sep 4, 2008

rhettinger commented Sep 4, 2008

jcea commented Sep 4, 2008

ncoghlan commented Sep 4, 2008

jcea commented Sep 4, 2008

smontanaro commented Sep 5, 2008

jcea commented Sep 5, 2008

brettcannon commented Sep 5, 2008

Deprecate bsddb for removal in 3.0 #48019

Deprecate bsddb for removal in 3.0 #48019

Comments

brettcannon commented Sep 3, 2008

brettcannon commented Sep 3, 2008

benjaminp commented Sep 3, 2008

smontanaro commented Sep 3, 2008

benjaminp commented Sep 3, 2008

rhettinger commented Sep 3, 2008

benjaminp commented Sep 3, 2008

brettcannon commented Sep 3, 2008

rhettinger commented Sep 3, 2008

brettcannon commented Sep 4, 2008

rhettinger commented Sep 4, 2008

warsaw commented Sep 4, 2008

smontanaro commented Sep 4, 2008

rhettinger commented Sep 4, 2008

jcea commented Sep 4, 2008

ncoghlan commented Sep 4, 2008

jcea commented Sep 4, 2008

smontanaro commented Sep 5, 2008

jcea commented Sep 5, 2008

brettcannon commented Sep 5, 2008