Weird bug. Attached are two files: wow.py and wy.py
When wow.py is executed - it fails with single ImportError.
But when wy.py is executed (which is a simple "import wow" statement) it fails with one ImportError and TypeError exceptions.

...one ImportError and *two* TypeError exceptions.

>py wy.py
Traceback (most recent call last):
  File "wy.py", line 1, in <module>
    import wow
  File "E:\scons\wow.py", line 12, in <module>
    import fail
ImportError: No module named fail
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "E:\scons\wow.py", line 8, in _clean
    cleanlist = [ c for c in _Cleanup if c ]
TypeError: 'NoneType' object is not iterable
Error in sys.exitfunc:
Traceback (most recent call last):
  File "C:\Python27\lib\atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "E:\scons\wow.py", line 8, in _clean
    cleanlist = [ c for c in _Cleanup if c ]
TypeError: 'NoneType' object is not iterable

It's because _Cleanup is None (as all global variables) when atexit registered functions executed. Don't use globals in such code.

But why _Cleanup is not None when wow.py is executed standalone?

When you run wy.py the wow module gets partially imported, and then garbage collected because it fails to import successfully.  The destructor for the module replaces values in the module's __dict__ with None.  So when the cleanup function runs you get the unexpected error.

When you run wow.py directly, wow (i.e. the main module) will not be garbage collected, so _Cleanup is never replaced by None.

This is not repeatable in Python 3. Is it possible to fix it for Python 2 as well?

Is it possible to postpone registration until the import is finished successfully? Or at least give atexit handler a chance to run before global variable stack is purged? Or destroy atexit handler if its namespace is purged?

Infortunately, the behaviour is the same in 2.7 and 3.2, and only changes with 3.3, which means that this is probably related to the new implementation of the import system.
This won't be backported.

[It would be interesting to understand why there is a difference, btw... is a reference to the module held somewhere?]

A possible workaround for such module-level calls atexit.register() is to ensure that used globals are captured in the function namespace, a bit like some __del__ methods:

def _clean(_Cleanup=_Cleanup):
    ....

OK, found the difference between 3.2 and 3.3:
The ImportError exception is still alive when atexit handlers are called, with its __traceback__, and all local variables in Python frames.
And since 3.3 the import system is built with Python functions...

More exactly, I could find the incomplete 'wow' module in sys.last_value.__traceback__.tb_next.tb_frame.f_back.f_back.f_back.f_locals['module']

But all this is not really related to the current issue.
Things should be better in the future, when modules are cleared with true garbage collection.

> Things should be better in the future, when modules are cleared with true 
> garbage collection.

When is this future of which you speak?

I am not sure whether it would affect performance, but a weakrefable subclass of dict could be used for module dicts.  Then the module destructor could just save the module's dict in a WeakValueDictionary keyed by the id (assuming we are not yet shutting down).  At shutdown the saved module dicts could be purged by replacing all values with None.

Or maybe something similar is possible without using a dict subclass.

See issue812369 for the shutdown procedure and modules cleanup.

> See issue812369 for the shutdown procedure and modules cleanup.

I am aware of that issue, but the original patch is 9 years old.  Which is why I ask if/when it will actually happen.

I don't know, you should probably ask there. One blocker is to make builtin and extension modules participate in GC, search "pep3121" to see the many intermediate issues.

Perhaps the simplest thing would be to stop doing anything special when a module is garbage collected: the garbage collector can take care of any orphaned ref-cycles involving the module dict.  Then at shutdown the remaining modules in sys.modules could have their dicts "purged" in the old way.

This would be orthogonal to issue812369.  In fact Armin's original post says that this is a change worth investigating, though his patch does not do it.

Since this was fixed since 3.3, is there a reason why this issue is still open?


I don't get the error anymore:

C:\Users\User\src\cpython-dev>python.bat wow.py
Running Release|x64 interpreter...
Traceback (most recent call last):
  File "C:\Users\User\src\cpython-dev\wow.py", line 12, in <module>
    import fail
ModuleNotFoundError: No module named 'fail'

C:\Users\User\src\cpython-dev>python.bat wy.py
Running Release|x64 interpreter...
Traceback (most recent call last):
  File "C:\Users\User\src\cpython-dev\wy.py", line 1, in <module>
    import wow
  File "C:\Users\User\src\cpython-dev\wow.py", line 12, in <module>
    import fail
ModuleNotFoundError: No module named 'fail'