classification
Title: Change in error when sys.path contains a nonexistent folder (importlib)
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, eric.smith, eric.snow, pitrou, python-dev, takluyver
Priority: normal Keywords:

Created on 2012-05-17 23:35 by takluyver, last changed 2012-08-20 03:18 by python-dev. This issue is now closed.

Messages (4)
msg161025 - (view) Author: Thomas Kluyver (takluyver) * Date: 2012-05-17 23:35
I've come across a difference from 3.2 to 3.3 while running the IPython test suite. It occurs when a directory on sys.path has been used for an import, then deleted without being removed from sys.path. Previous versions of Python ignore the nonexistent folder, but in 3.3 a FileNotFound error appears.

This might be by design (errors should never pass silently), but I haven't found it mentioned in the What's New for 3.3.

$ cat import.py
import sys, os, shutil
os.mkdir('foo')
with open('foo/bar.py', 'w'):
    pass
sys.path.insert(0, 'foo/')
import bar   # Caches a FileFinder for foo/
shutil.rmtree('foo')
import random   # Comes later on sys.path

$ python3.2 import.py

$ python3.3 import.py
Traceback (most recent call last):
  File "import.py", line 8, in <module>
    import random
  File "<frozen importlib._bootstrap>", line 1162, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1124, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1078, in _find_module
  File "<frozen importlib._bootstrap>", line 927, in find_module
  File "<frozen importlib._bootstrap>", line 973, in find_module
  File "<frozen importlib._bootstrap>", line 1005, in _fill_cache
FileNotFoundError: [Errno 2] No such file or directory: 'foo/'

The last entry in that traceback is calling "_os.listdir(path)".
msg161026 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-05-18 00:15
The caching mechanism is going to check for changes in the directory.  So the recommendation is to clear the cache manually:

<quote>
  The default finders used by import now utilize a cache of what is contained
  within a specific directory. If you create a Python source file or sourceless
  bytecode file, make sure to call importlib.invalidate_caches() to clear out
  the cache for the finders to notice the new file.
</quote>

(see http://docs.python.org/dev/whatsnew/3.3.html#porting-python-code)

That section doesn't really talk about the directory side of things, but calling importlib.invalidate_caches() works great to eliminate the error in your example.
msg161027 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2012-05-18 00:16
...rather, the error caused by your example.  :)
msg168616 - (view) Author: Roundup Robot (python-dev) Date: 2012-08-20 03:18
New changeset bfd04bfb55c5 by Nick Coghlan in branch 'default':
Close #14846: Handle a sys.path entry going away
http://hg.python.org/cpython/rev/bfd04bfb55c5
History
Date User Action Args
2012-08-20 03:18:39python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg168616

resolution: fixed
stage: resolved
2012-05-18 01:48:50eric.smithsetnosy: + eric.smith
2012-05-18 00:16:58eric.snowsetmessages: + msg161027
2012-05-18 00:15:49eric.snowsetnosy: + brett.cannon, pitrou, eric.snow
messages: + msg161026
2012-05-17 23:35:08takluyvercreate