classification
Title: Destructor behavior faulty
Type: behavior Stage: test needed
Components: Interpreter Core Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: module shutdown procedure based on GC
View: 812369
Assigned To: Nosy List: alanmcintyre, georg.brandl, ggenellina, pitrou, sf-robot, wrogner
Priority: normal Keywords:

Created on 2007-05-12 20:41 by wrogner, last changed 2009-03-31 13:17 by pitrou. This issue is now closed.

Messages (12)
msg31992 - (view) Author: Wolf Rogner (wrogner) Date: 2007-05-12 20:41
I tried example 11.4. from bytesofpython (by C.H. Swaroop).
Example works fine.

Added a new Person instance 'wolf' -> Program terminated with:
Exception exceptions.AttributeError: "'NoneType' object has no attribute 'population'" in <bound method Person.__del__ of <__main__.Person instance at 0xb7d48d6c>> ignored

added print list(globals()) ->
['kalam', '__builtins__', '__file__', 'DBGPHideChildren', 'swaroop', 'Person', 'wolf', '__name__', '__doc__']

changed wolf to aaa:

print list(globals()) ->
['aaa', 'kalam', '__builtins__', '__file__', 'DBGPHideChildren', 'swaroop', 'Person', '__name__', '__doc__']

Please note the position of 'aaa' at the beginning of the list, before 'Person'. With 'wolf' being after 'Person'.

If the destructing code removes items in this order, no wonder I get an error.

Person should not get deleted if refcount is still > 0.

Wolf Rogner
msg31993 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-05-12 23:36
Could you post the code for your entire script? It makes it a lot easier to figure out what's going on.
msg31994 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-05-14 15:38
I took the example mentioned from here: 
    http://www.python.org/doc/essays/cleanup/

and added this line to the end:

wolf = Person('wolf')

and it gives the reported error.  Here is a minimal snippet that produces the same error when executed as the top-level module:

class Person:
        population = 0
        def __del__(self):
                Person.population -= 1

wolf = Person()


This appears to be consistent with the behavior described here:
     http://www.python.org/doc/essays/cleanup/

While I understand that cleaning up a module at exit time is probably not an easy thing to make arbitrarily smart, this behavior seems a little too not-smart to me.  It seems like it's not all that hard to get bitten by  it, and the error makes no sense unless you're familiar with the module cleanup algorithm.

For what it's worth, I offer to help make module cleanup a little smarter, although I may not be able to spend much time on it until I finish some things I'm already committed to do.
msg31995 - (view) Author: Gabriel Genellina (ggenellina) Date: 2007-05-15 18:44
FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.
msg31996 - (view) Author: Gabriel Genellina (ggenellina) Date: 2007-05-16 11:19
FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.
msg31997 - (view) Author: Gabriel Genellina (ggenellina) Date: 2007-05-16 22:06
FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.
msg31998 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-05-16 22:58
I tried out a simple change (to the trunk) in _PyModule_Clear to prevent this particular problem.  Between the "remove everything beginning with an underscore" and the "remove everything except __builtins__" steps, I added a step to remove all instance objects in the module's dictionary.  It appears to stop this problem and passes the regression test suite.  I can post it as a patch if this seems like a reasonable change to make. 

Also, I noticed that earlier I posted the wrong link for the location of the example code; it should have been:
     http://www.dpawson.co.uk/bop/byteofpython_120.txt
msg31999 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-05-17 08:15
Alan: you should bring that up on python-dev.
msg32000 - (view) Author: SourceForge Robot (sf-robot) Date: 2007-05-28 02:20
This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).
msg32001 - (view) Author: Alan McIntyre (alanmcintyre) * (Python committer) Date: 2007-06-06 16:38
Brought this up on python-dev: http://mail.python.org/pipermail/python-dev/2007-May/073329.html

Since there is some interest in making the change Armin suggested, I suggest re-opening this item so that it doesn't get overlooked/forgotten.
msg84772 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-31 13:15
Armin's proposal is in #812369, closing this bug.
(it is not obvious Armin's patch is enough to solve the problem at hand,
but the problem is well-known anyway and there are certainly other bug
entries pointing to it :-))
msg84774 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-03-31 13:17
By the way, an easy way to fix it would probably to rewrite the
destructor in the following way (haven't tested):

    def __del__(self):
        self.__class__.population -= 1
History
Date User Action Args
2009-03-31 13:17:18pitrousetmessages: + msg84774
2009-03-31 13:16:00pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg84772

superseder: module shutdown procedure based on GC
resolution: duplicate
2009-03-31 00:47:36ajaksu2setstage: test needed
type: behavior
versions: + Python 3.1, Python 2.7
2007-05-12 20:41:54wrognercreate