This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Failed imports clean up module, but not sub modules
Type: behavior Stage: resolved
Components: Interpreter Core Versions:
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Calvin.Spealman, eric.snow, r.david.murray
Priority: normal Keywords:

Created on 2011-07-13 19:44 by Calvin.Spealman, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Messages (6)
msg140298 - (view) Author: Calvin Spealman (Calvin.Spealman) Date: 2011-07-13 19:44
I came across this behavior when it was related to the shadowing of a small typo bug and took me forever to find. In my case, the original error was shadowed in a way that is unrelated to this bug, but lead to the module being imported twice (because it was removed when it failed the first time) and then the second import caused a completely unexpected error, because the state of the submodules conflicted with the import-time logic of the top-level package.

I think when a module fails to load, all of its sub-modules should be removed from sys.modules, not just itself.

calvin@willow-delta:/tmp$ mkdir foo/
calvin@willow-delta:/tmp$ cat >> foo/__init__.py
import foo.bar
1/0
calvin@willow-delta:/tmp$ cat >> foo/bar.py
name = "bar"
calvin@willow-delta:/tmp$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "foo/__init__.py", line 2, in <module>
    1/0
ZeroDivisionError: integer division or modulo by zero
>>> import sys
>>> sys.modules['foo.bar']
<module 'foo.bar' from 'foo/bar.py'>
>>>
msg140300 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-07-13 19:51
That can't be done.  If an import fails, it fails.  But if it succeeds, the module is loaded, and there is no reason to unload it.  After all, import in python is idempotent (doing it a second time has no effect other than adding the name to the local namespace).  So the import in the module that failed to load may not be the first, and unloading it would break other modules using it.
msg140316 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2011-07-13 22:54
But it is curious that the submodules were added to sys.modules even though the package failed to import.
msg140318 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-07-13 23:13
Yes, that's exactly my point.  The loading of a module into sys.modules is a separate issue from the creation of a pointer to that module in the local name space.  Once the former succeeds, it has succeeded and won't be done again.  When the module that did the import fails to complete, *it* is cleaned up, which cleans up the pointer in the local name space, but that doesn't (by design) affect sys.modules.

It's not all that dissimilar to what would happen if you had top-level code in your module that opened and wrote to a file.  If an exception later in the module code causes it to fail to load, the file it created would not be deleted.
msg140320 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-07-14 01:28
In fact, to hopefully make perfectly clear what is going on here, let me demonstrate that *any* executable statement in the module that fails to load is executed if it occurs before the error that causes the load failure:

    rdmurray@hey:~/python/p32>cat temp.py
    import os

    print('foo:', hasattr(os, 'foo'))

    try:
        import temp2
    except AttributeError:
        print('attribute error')

    print('temp2:', 'temp2' in globals())
    print('foo:', hasattr(os, 'foo'))
    rdmurray@hey:~/python/p32>cat temp2.py
    import os

    os.foo = 2
    os.bar

    rdmurray@hey:~/python/p32>./python temp.py
    foo: False
    attribute error
    temp2: False
    foo: True
msg140321 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2011-07-14 03:20
Yeah, I missed the line where he imported foo.bar in "foo/__init__.py", and thought something else was going on.
  
"foo/__init__.py" does not have to finish importing (just has to be on sys.modules) at the point foo.bar is imported.  So foo.bar would be completely imported before the "1/0" expression was evaluated, and you are exactly right, David.

Sorry for adding to the confusion.
History
Date User Action Args
2022-04-11 14:57:19adminsetgithub: 56763
2011-07-14 03:20:15eric.snowsetmessages: + msg140321
2011-07-14 01:28:57r.david.murraysetmessages: + msg140320
2011-07-13 23:13:52r.david.murraysetmessages: + msg140318
2011-07-13 22:54:43eric.snowsetnosy: + eric.snow
messages: + msg140316
2011-07-13 19:51:37r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg140300

resolution: not a bug
stage: resolved
2011-07-13 19:44:22Calvin.Spealmancreate