classification
Title: ABC caches should use weak refs
Type: Stage:
Components: Versions: Python 3.0
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: georg.brandl Nosy List: georg.brandl, gvanrossum, loewis, twouters
Priority: normal Keywords: patch

Created on 2007-08-30 13:48 by gvanrossum, last changed 2007-10-23 06:27 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
abc_weakref_set.diff twouters, 2007-08-30 15:07
wr.diff georg.brandl, 2007-10-20 07:42
Messages (9)
msg55480 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-30 13:48
The various caches in abc.py should be turned into weak sets so that the
caches don't keep the subclass objects alive forever.
msg55483 - (view) Author: Thomas Wouters (twouters) * (Python committer) Date: 2007-08-30 15:07
Here's a working version of that idea, with a WeakSet implementation I
had lying around (but never really used.) It seems to work, and fixes
the refcount issues, but the WeakSet could do with some extra tests ;-)
msg55490 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-30 16:36
On 8/30/07, Thomas Wouters <report@bugs.python.org> wrote:
>
> Thomas Wouters added the comment:
>
> Here's a working version of that idea, with a WeakSet implementation I
> had lying around (but never really used.) It seems to work, and fixes
> the refcount issues, but the WeakSet could do with some extra tests ;-)

I was torturing the WeakSet implementation, but didn't get very far:

Python 3.0x (py3k, Aug 30 2007, 09:27:35)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from weakref import WeakSet as WS
[40407 refs]
>>> a = WS([1, 2, 3])
Fatal Python error: Cannot recover from stack overflow.
Aborted
msg55502 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-08-30 19:02
Guido van Rossum schrieb:
> Guido van Rossum added the comment:
> 
> On 8/30/07, Thomas Wouters <report@bugs.python.org> wrote:
>>
>> Thomas Wouters added the comment:
>>
>> Here's a working version of that idea, with a WeakSet implementation I
>> had lying around (but never really used.) It seems to work, and fixes
>> the refcount issues, but the WeakSet could do with some extra tests ;-)
> 
> I was torturing the WeakSet implementation, but didn't get very far:
> 
> Python 3.0x (py3k, Aug 30 2007, 09:27:35)
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from weakref import WeakSet as WS
> [40407 refs]
>>>> a = WS([1, 2, 3])
> Fatal Python error: Cannot recover from stack overflow.
> Aborted

The update() method can not be implemented in terms of converting the
argument to a WeakSet, since that leads to infinite recursion on creation
(as to why you get this fatal error, not a RuntimeError, no idea).

Also, the fact that the operator-versions of set operations only support
other sets as their second operands, while the method-versions support any
iterable, is not preserved.

Georg
msg55504 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2007-08-30 19:10
The fatal error is raised if the stack overflows in the process of
handling RuntimeError. In certain cases, the C algorithms won't bail out
if a RuntimeError is raised, and just catch it. If that then causes
another stack overflow, Python gives up. One such case is an overflow
occuring during a comparison operation of a dictionary lookup, IIRC.
msg55511 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-08-30 20:45
Georg, would you have time to submit an improved patch?
msg56602 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-10-20 07:42
Attaching a new patch; this one passes regrtest -R:: of test_io without
apparent reference leaks.
msg56666 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2007-10-23 03:49
Thanks Georg; please check it in!
msg56671 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-10-23 06:27
Committed r58602.
History
Date User Action Args
2007-10-23 06:27:05georg.brandlsetstatus: open -> closed
messages: + msg56671
2007-10-23 03:49:08gvanrossumsetassignee: gvanrossum -> georg.brandl
resolution: accepted
messages: + msg56666
2007-10-20 07:43:00georg.brandlsetfiles: + wr.diff
assignee: georg.brandl -> gvanrossum
messages: + msg56602
2007-09-17 07:44:40jafosetpriority: normal
assignee: georg.brandl
2007-09-02 20:11:25loewissetkeywords: + patch
2007-08-30 20:45:45gvanrossumsetmessages: + msg55511
2007-08-30 19:10:38loewissetnosy: + loewis
messages: + msg55504
2007-08-30 19:02:39georg.brandlsetnosy: + georg.brandl
messages: + msg55502
2007-08-30 16:36:25gvanrossumsetmessages: + msg55490
2007-08-30 15:07:19twouterssetfiles: + abc_weakref_set.diff
nosy: + twouters
messages: + msg55483
2007-08-30 13:48:44gvanrossumcreate