classification
Title: dbm.open(..., flag="n") does not work and does not give a warning
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Claudiu.Popa, berker.peksag, python-dev, r.david.murray, serhiy.storchaka, sonyachiko
Priority: normal Keywords: patch

Created on 2013-05-23 00:39 by sonyachiko, last changed 2014-06-10 19:22 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
dbmopen.py berker.peksag, 2013-05-31 10:49
test_dbm.py Claudiu.Popa, 2013-11-01 22:17
dbm_dumb_open.patch Claudiu.Popa, 2013-11-01 22:17 review
issue18039.patch Claudiu.Popa, 2014-03-11 21:00 Use `with` in test and minor doc fixes review
issue18039_1.patch Claudiu.Popa, 2014-06-09 19:45 review
issue18039_2.patch Claudiu.Popa, 2014-06-10 08:47 review
issue18039_3.patch Claudiu.Popa, 2014-06-10 15:59 review
Messages (15)
msg189841 - (view) Author: Sashko Kopyl (sonyachiko) Date: 2013-05-23 00:39
I use Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)] on win32.

dbm.open(..., flag="n") is supposed to "Always create a new, empty database, open for reading and writing".
In Windows case it does not. So I cannot empty a dbm database when I need it.

http://docs.python.org/3/library/dbm.html#module-dbm.dumb
Here it is written:
"The optional flag argument is currently ignored; the database is always opened for update, and will be created if it does not exist."

If it is ignored, there should be at least a warning. How am I supposed to now that this is a known bug?
msg190401 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2013-05-31 10:49
I can't reproduce it on Windows 7 with Python 3.3.2.

Attaching my test script.

Here's the output:

C:\Python33>python.exe -V
Python 3.3.2

C:\Python33>python.exe dbmopen.py
<dbm.dumb._Database object at 0x027C15B0>
bzdew.dat exists? True
bzdew.dir exists? True
b'hello' b'there'

Could you run it and paste here the output?
msg190403 - (view) Author: Sashko Kopyl (sonyachiko) Date: 2013-05-31 11:53
Here is the output.


*** Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600 32 bit (Intel)] on win32. ***
*** Remote Python engine  is active ***
>>> 
*** Remote Interpreter Reinitialized  ***
>>> 
<dbm.dumb._Database object at 0x02B95210>
yoqaA.dat exists? True
yoqaA.dir exists? True
b'hello' b'there'
>>> 


I would like to focus your attention, that flag "n" creates a database, but does not overwrite it once it is created. So in Windows case there is no difference between "c" and "n" flag.
You can have a look at this link where it was originally discussed.
http://stackoverflow.com/questions/16647131/how-to-empty-dbm-file-in-python-efficiently
msg201944 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-11-01 22:17
Here's a patch which adds support for the `n` value of the flag. It makes the dbm consistent across implementations and it's thus more reliable.
Also, attached a script which replicates the problem for Windows (where dbm.dumb is used).


My output (after a couple of runs):


D:\Projects\snippets\DBM>py -3 a.py u7
test
[b'u3', b'u2', b'u5', b'u4', b'u7', b'u6']
test.dat exists? True
test.dir exists? True
b'u3' b'there'
b'u2' b'there'
b'u5' b'there'
b'u4' b'there'
b'u7' b'there'
b'u6' b'there'
msg201945 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-11-01 22:17
And the patch.
msg217249 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-04-27 10:14
Can anyone review this patch?
msg219553 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-06-02 07:47
Serhiy, could you please have a look at this patch? Given the fact that you committed my last dbm patch, I hope you have a couple of minutes to have a look at this one as well.
msg220111 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-06-09 19:10
With this patch 2 of 4 modes ('c' and 'n') work as for other dbm modules. So it would be better to document that 'r' and 'w' work nonstandard, than document that 'n' is not ignored.

And while you are here, may be add a warning in 'r' and 'w' mode when the file does not exist?
msg220113 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-06-09 19:45
Thanks, Serhiy. Here's the new version of the patch. Hope that the warning message is clear enough.
msg220153 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-06-10 08:47
Thanks for the reviews, Serhiy. Here's the new version of the patch.
msg220184 - (view) Author: Roundup Robot (python-dev) Date: 2014-06-10 18:30
New changeset 3f944f44ee41 by Serhiy Storchaka in branch 'default':
Issue #18039: dbm.dump.open() now always creates a new database when the
http://hg.python.org/cpython/rev/3f944f44ee41
msg220187 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-06-10 18:35
Committed original Claudiu's patch (issue18039.patch) with minor changes. For warnings I opened separate issue (issue21708) because patch becomes too large.

Thank you Claudiu for your patch.
msg220188 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-10 18:37
Isn't this change going to cause unexpected data loss for (possibly mythical) people depending on the existing behavior?  At an absolute minimum it needs an entry in the What's New porting section, but I'm wondering if a deprecation period is more appropriate.
msg220190 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-06-10 18:49
I think the chance of this is pretty small. I can imagine that some people use default, 'r' or 'w' mode unintentionally, and all works for them even when database is missed (for example our tests do this). But it is very unlikely that someone use the 'n' mode and expect that database will be not cleaned.
msg220192 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2014-06-10 19:22
Yeah, hopefully you are right.  (I didn't mean to reopen the issue, by the way).
History
Date User Action Args
2014-06-10 19:22:15r.david.murraysetstatus: open -> closed
resolution: fixed
messages: + msg220192

stage: patch review -> resolved
2014-06-10 18:50:22serhiy.storchakasetresolution: fixed -> (no value)
stage: resolved -> patch review
2014-06-10 18:49:37serhiy.storchakasetresolution: fixed
messages: + msg220190
stage: patch review -> resolved
2014-06-10 18:37:10r.david.murraysetstatus: closed -> open

nosy: + r.david.murray
messages: + msg220188

resolution: fixed -> (no value)
stage: resolved -> patch review
2014-06-10 18:35:02serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg220187

stage: patch review -> resolved
2014-06-10 18:30:06python-devsetnosy: + python-dev
messages: + msg220184
2014-06-10 15:59:16Claudiu.Popasetfiles: + issue18039_3.patch
2014-06-10 08:47:46Claudiu.Popasetfiles: + issue18039_2.patch

messages: + msg220153
2014-06-10 08:14:36serhiy.storchakasettype: behavior -> enhancement
2014-06-09 19:45:24Claudiu.Popasetfiles: + issue18039_1.patch

messages: + msg220113
2014-06-09 19:10:01serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg220111
2014-06-02 07:47:58Claudiu.Popasetmessages: + msg219553
2014-06-02 07:44:39Claudiu.Popasetnosy: + serhiy.storchaka
2014-04-27 10:14:18Claudiu.Popasetmessages: + msg217249
2014-03-11 23:12:15berker.peksagsetstage: test needed -> patch review
2014-03-11 21:00:05Claudiu.Popasetfiles: + issue18039.patch
versions: + Python 3.5, - Python 3.3
2013-11-01 22:17:23Claudiu.Popasetfiles: + dbm_dumb_open.patch
keywords: + patch
messages: + msg201945
2013-11-01 22:17:00Claudiu.Popasetfiles: + test_dbm.py
nosy: + Claudiu.Popa
messages: + msg201944

2013-05-31 11:53:14sonyachikosetmessages: + msg190403
2013-05-31 10:49:09berker.peksagsetfiles: + dbmopen.py

nosy: + berker.peksag
messages: + msg190401

stage: test needed
2013-05-23 00:39:58sonyachikocreate