This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author gvanrossum
Recipients Arfrever, Jim.Jewett, asvetlov, gregory.p.smith, gvanrossum, ncoghlan, pitrou, r.david.murray, rhettinger, skrah, vstinner
Date 2012-04-01.23:44:29
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <CAP7+vJJGggBgS4rL5YeJAnz-OEiSNj_mMmCBKVxdteZL7mZbcg@mail.gmail.com>
In-reply-to <1333313890.64.0.879232167517.issue14417@psf.upfronthosting.co.za>
Content
On Sun, Apr 1, 2012 at 1:58 PM, Raymond Hettinger
<report@bugs.python.org> wrote:
[...]
> I'm wary of the original RuntimeError patch because
[...]

I had retorts to most of what you wrote, but decided not to post them.
Instead, I really want to have a serious discussion about this issue,
without falling back on "if it ain't broke don't fix it". There was
definitely *something* wrong with the original code: there was a known
crasher and it had XXX comments in it, and some folks cared enough to
attempt to fix it.

In the end I see this more as a matter of clarifying the semantics and
limitations of dict operations: *should* the language guarantee that
the built-in dict implementation supports correct lookup even if the
in the middle of the lookup the hash table is resized by another
thread? Is this a reasonable guarantee to impose on all Python
implementations? Because I don't see how this can be guaranteed
without a lock; but I haven't tried very hard, so this is not a
rhetorical question.

- - -

On Sun, Apr 1, 2012 at 4:01 PM, Nick Coghlan <report@bugs.python.org> wrote:
> A thought prompted by Raymond's comment: did we ever try just protecting
> the retry as a recursive call? If we can stop the stack blowing up, it
> seems to me we'd get the best of both worlds (that is, crashes become
> RuntimeError, but naive multi-threaded code is unaffected).

Hm... We considered a variety of fixes in
http://bugs.python.org/issue14205 but I don't think we tried that.
(The nastiness is that it's strictly a C-level recursion -- lookdict()
calls lookdict().)

Another thing we didn't try was manually eliminating the tail
recursion in lookdict() using a goto (and perhaps some variable
assignments). I don't see the potential for an infinite loop as a
problem, since one could be created just as easily by putting "while
True: pass" inside a user-defined __hash__().

PS. I'm not surprised your originall hammer_dict.py didn't provoke the
problem -- it only overrides __hash__(), but lookdict() never calls
that. It only calls the compare function; the hash is computed once
and passed into it.

Finally, the guard in lookdict() looks fishy to me:

  if (ep0 == mp->ma_table && ep->me_key == startkey) {

Can't this evaluate to true when the dict has shrunk dramatically in
size? I think an extra test for "mask == mp->ma_mask" ought to be
added.
History
Date User Action Args
2012-04-01 23:44:30gvanrossumsetrecipients: + gvanrossum, rhettinger, gregory.p.smith, ncoghlan, pitrou, vstinner, Arfrever, r.david.murray, asvetlov, skrah, Jim.Jewett
2012-04-01 23:44:30gvanrossumlinkissue14417 messages
2012-04-01 23:44:29gvanrossumcreate