Title: Local files shadow system modules, even from system modules
Type: security Stage: resolved
Components: Interpreter Core Versions: Python 3.4
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: drt24, ncoghlan
Priority: normal Keywords:

Created on 2014-08-08 09:32 by drt24, last changed 2014-08-08 11:48 by ncoghlan. This issue is now closed.

Messages (2)
msg225065 - (view) Author: Daniel Thomas (drt24) Date: 2014-08-08 09:32
In Python 3.4 (but not 3.2 or 2.7) when a system module does an import then files with the same name in the directory of the original python script which match that name are used.

With a directory containing:
    #!/usr/bin/env python3
    from collections import OrderedDict
    print('do stuff')
    #!/usr/bin/env python3

Running will cause:
Traceback (most recent call last):
  File "./", line 4, in <module>
    from collections import OrderedDict
  File "/usr/lib/python3.4/collections/", line 11, in <module>
    from operator import itemgetter as _itemgetter, eq as _eq
ImportError: cannot import name 'itemgetter'

While is perfectly innocent it is in the same directory as the nasty and makes no reference at all to but when 'collections' is imported it imports from operator which is resolved to in the local directory

This is a security vulnerability because it is possible to verify that a python script is safe to run by reading its code and then on running it find that other code is implicitly loaded by the system libraries which is never referenced in the original file or part of any of the standard libraries.

It is also rather confusing but a related issue is already filed for that in issue21202.
This is similar to the standard name shadowing trap but now applies to other files in the source directory in a way which it didn't in previous versions of python. I suspect this was introduced in python 3.3 through changes to the import system and becoming optional but I don't have a 3.3 install to check that with.

sys.path here is: ['/auto/homes/drt24/pythontest', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
Running Python 3.4.0-0ubuntu2 on Ubuntu 14.04.1 LTS with Linux 3.13.0-32-generic
msg225066 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2014-08-08 11:48
This is not new behaviour. It's just normal system shadowing:

$ python 
Traceback (most recent call last):
  File "", line 1, in <module>
    import collections
  File "/usr/lib64/python2.7/", line 9, in <module>
    from operator import itemgetter as _itemgetter, eq as _eq
ImportError: cannot import name itemgetter

Avoiding that behaviour is what the new isolated mode is for:

$ ~/devel/py34/python -I 
Date User Action Args
2014-08-08 11:48:58ncoghlansetstatus: open -> closed

nosy: + ncoghlan
messages: + msg225066

resolution: not a bug
stage: resolved
2014-08-08 09:32:09drt24create