Author lehmannro
Recipients lehmannro, zhigang
Date 2009-09-16.17:33:49
SpamBayes Score 1.31937e-08
Marked as misclassified No
Message-id <>
If I understand you correctly, your proposal is the following: use
Shelf.cache to cache *all* objects instead of only keeping live
references. Your patch retains the cache forever instead of purging it
on sync. (All these changes only apply with writeback=True, which you
enabled by default; nothing changes with writeback=False.)
This speeds up *repeated* reads/writes as they are kept in-memory
instead of querying the-- probably slow --database every time.

I do not think this is a feasible solution for two reasons:
(1) writeback was never intended to do such thing. It was introduced as
a solution to "make shelve less surprising." If you remove its
sync-on-close characteristics, shelve is as surprising as before. See
(2) If you intend to implement caching on *any* database I'd suggest not
using shelve for that matter. Using it for serialization is all okay but
I'd rather add an additional layer of indirection to implement caching
strategies if you want to do that The Right Way. (Shelf.cache is really
only a map of objects that were touched during runtime.)

I'm -1 on this patch but generally +1 on a generic caching wrapper.

The error you describe later on appears because Python is already
tearing down when gc'ing the Shelf (calling __del__ -> close -> sync).
With writeback=True this currently tries to pickle the cache again which
emits the error you observed. This should be handled in a separate issue.
Date User Action Args
2009-09-16 17:33:51lehmannrosetrecipients: + lehmannro, zhigang
2009-09-16 17:33:51lehmannrosetmessageid: <>
2009-09-16 17:33:49lehmannrolinkissue5483 messages
2009-09-16 17:33:49lehmannrocreate