This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Shelve module writeback parameter does not act as advertised
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: aleax, jherskovic, lehmannro, r.david.murray
Priority: normal Keywords: easy, patch

Created on 2009-04-14 17:03 by jherskovic, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue5754.patch lehmannro, 2009-09-17 13:09 patch to Lib/test/test_shelve.py and Doc/library/shelve.rst, trunk
Messages (3)
msg85971 - (view) Author: Jorge Herskovic (jherskovic) Date: 2009-04-14 17:03
The shelve module documentation states that "by default, mutations to
persistent-dictionary mutable entries are not automatically written
back. If the optional writeback parameter is set to True, all entries
accessed are cached in memory, and written back at close time..."
however the implementation's __setitem__ is the following:
    def __setitem__(self, key, value):
        if self.writeback:
            self.cache[key] = value
        f = StringIO()
        p = Pickler(f, self._protocol)
        p.dump(value)
        self.dict[key] = f.getvalue()

which maintains the cache correctly but writes back to the disk on every
operation, violating the writeback documentation. Changing it to 
    def __setitem__(self, key, value):
        if self.writeback:
            self.cache[key] = value
        else:
            f = StringIO()
            p = Pickler(f, self._protocol)
            p.dump(value)
            self.dict[key] = f.getvalue()

seems to match the documentation's intent.

(First report, sorry for any formatting/style issues!)
msg92761 - (view) Author: Robert Lehmann (lehmannro) * Date: 2009-09-17 13:09
I think you're misquoting Python's shelve module documentation in your
first sentence. The documentation says:
"By default modified objects are written only when assigned to the shelf
[...]. If the optional writeback parameter is set to True, all entries
accessed are cached in memory, and written back at close time [...]."

The emphasis should be on the word "only:" it does *always* write to the
database when assigned to the shelf but, iff writeback=True, *also* to
the cache.

Also consider the consequences of *only* caching keys:
(a) __contains__ and has_key have to consult the dict and the cache for
membership tests.
(b) keys and __len__ need to compute a union of both sources.
(c) __delitem__ is no longer guaranteed to fail on the cache if it
failed for the database.

I admit the docs could spell this out more clearly. I attached a patch
ensuring the behaviour I described in a test and updating the docs.

(Note: shelve is no extension module -- it's part of the stdlib. Patch
applies to 3.x as well.)
msg99189 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-02-11 01:50
Thanks for the patch.  I applied the doc patch and a slightly simplified version of the test in r78141 (we tend to just let errors bubble up rather than code explicit Fails, since as often as not if you provide a specific message it turns out to be wrong when something breaks the test).
I'll merge the change to the other branches as well.
History
Date User Action Args
2022-04-11 14:56:47adminsetgithub: 50004
2010-02-11 01:50:33r.david.murraysetstatus: open -> closed
priority: normal

assignee: aleax -> r.david.murray
versions: + Python 3.1, Python 2.7, Python 3.2
nosy: + r.david.murray

messages: + msg99189
resolution: accepted
stage: test needed -> resolved
2009-09-17 13:09:17lehmannrosetfiles: + issue5754.patch

nosy: + lehmannro
messages: + msg92761

components: + Library (Lib), - Extension Modules
keywords: + patch
2009-04-22 05:07:11ajaksu2setkeywords: + easy
stage: test needed
2009-04-14 20:56:43rhettingersetassignee: aleax

nosy: + aleax
2009-04-14 17:03:23jherskoviccreate