Author dmugtasimov
Recipients dmugtasimov, docs@python
Date 2013-01-08.08:14:30
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1357632871.49.0.196561776247.issue16891@psf.upfronthosting.co.za>
In-reply-to
Content
http://docs.python.org/2/tutorial/modules.html should be rewritten.
AS IS
6.1.2. The Module Search Path

When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

TO BE
6.1.2. The Module Search Path

When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it looks in the containing package (the package of which the current module is a submodule). If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:

------
Note that now "6.1.2. The Module Search Path" and "6.4.2. Intra-package References" are contradictary since in 6.4.2 it is said: "In fact, such references are so common that the import statement first looks in the containing package before looking in the standard module search path.", but this is not reflected in 6.1.2.

------
EXAMPLE (for more information see  http://stackoverflow.com/questions/14183541/why-python-finds-module-instead-of-package-if-they-have-the-same-name#comment19687166_14183541 ):
/home/dmugtasimov/tmp/name-res3/xyz
    __init__.py
    a.py
    b.py
    t.py
    xyz.py

Files init.py, b.py and xyz.py are empty
File a.py:

import os, sys
ROOT_DIRECTORY = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if not sys.path or ROOT_DIRECTORY not in sys.path:
    print 'sys.path is modified in a.py'
    sys.path.insert(0, ROOT_DIRECTORY)
else:
    print 'sys.path is NOT modified in a.py'

print 'sys.path:', sys.path
print 'BEFORE import xyz.b'
import xyz.b
print 'AFTER import xyz.b'

File t.py:

import os, sys
ROOT_DIRECTORY = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
if not sys.path or ROOT_DIRECTORY not in sys.path:
    print 'sys.path is modified in t.py'
    sys.path.insert(0, ROOT_DIRECTORY)
else:
    print 'sys.path is NOT modified in t.py'

import xyz.a

Run:

python a.py

Output:

    sys.path is modified in a.py
    sys.path: ['/home/dmugtasimov/tmp/name-res3', '/home/dmugtasimov/tmp/name-res3/xyz',
     '/usr/local/lib/python2.7/dist-packages/tornado-2.3-py2.7.egg',
     '/home/dmugtasimov/tmp/name-res3/xyz', '/usr/lib/python2.7',
     '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk',
     '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload',
     '/usr/local/lib/python2.7/dist-packages',
     '/usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info',
     '/usr/lib/python2.7/dist-packages',
     '/usr/lib/python2.7/dist-packages/PIL',
     '/usr/lib/python2.7/dist-packages/gst-0.10',
     '/usr/lib/python2.7/dist-packages/gtk-2.0',
     '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
    BEFORE import xyz.b
    AFTER import xyz.b

Run:

python -vv a.py

Output:

    import xyz # directory /home/dmugtasimov/tmp/name-res3/xyz
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__module.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    # /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    import xyz # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/b.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/bmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/b.py
    # /home/dmugtasimov/tmp/name-res3/xyz/b.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/b.py
    import xyz.b # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/b.pyc

Run:

python t.py

Output:

    sys.path is modified in t.py
    sys.path is NOT modified in a.py
    sys.path: ['/home/dmugtasimov/tmp/name-res3', '/home/dmugtasimov/tmp/name-res3/xyz',
     '/usr/local/lib/python2.7/dist-packages/tornado-2.3-py2.7.egg',
     '/home/dmugtasimov/tmp/name-res3/xyz', '/usr/lib/python2.7',
     '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk',
     '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload',
     '/usr/local/lib/python2.7/dist-packages',
     '/usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info',
     '/usr/lib/python2.7/dist-packages',
     '/usr/lib/python2.7/dist-packages/PIL',
     '/usr/lib/python2.7/dist-packages/gst-0.10',
     '/usr/lib/python2.7/dist-packages/gtk-2.0',
     '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
    BEFORE import xyz.b
    Traceback (most recent call last):
      File "t.py", line 9, in <module>
        import xyz.a
      File "/home/dmugtasimov/tmp/name-res3/xyz/a.py", line 11, in <module>
        import xyz.b
    ImportError: No module named b

Run:

python -vv t.py

Output:

    import xyz # directory /home/dmugtasimov/tmp/name-res3/xyz
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__module.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    # /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    import xyz # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/a.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/amodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/a.py
    # /home/dmugtasimov/tmp/name-res3/xyz/a.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/a.py
    import xyz.a # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/a.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/os.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/osmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/os.py
    # trying /home/dmugtasimov/tmp/name-res3/xyz/os.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/sys.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/sysmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/sys.py
    # trying /home/dmugtasimov/tmp/name-res3/xyz/sys.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyz.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyzmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyz.py
    # /home/dmugtasimov/tmp/name-res3/xyz/xyz.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/xyz.py
    import xyz.xyz # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/xyz.pyc
    #   clear[2] __file__
    #   clear[2] __package__
    #   clear[2] sys
    #   clear[2] ROOT_DIRECTORY
    #   clear[2] __name__
    #   clear[2] os
    sys.path is modified in t.py
    sys.path is NOT modified in a.py
    sys.path: ['/home/dmugtasimov/tmp/name-res3', '/home/dmugtasimov/tmp/name-res3/xyz',
     '/usr/local/lib/python2.7/dist-packages/tornado-2.3-py2.7.egg',
     '/home/dmugtasimov/tmp/name-res3/xyz', '/usr/lib/python2.7',
     '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk',
     '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload',
     '/usr/local/lib/python2.7/dist-packages',
     '/usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info',
     '/usr/lib/python2.7/dist-packages',
     '/usr/lib/python2.7/dist-packages/PIL',
     '/usr/lib/python2.7/dist-packages/gst-0.10',
     '/usr/lib/python2.7/dist-packages/gtk-2.0',
     '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
    BEFORE import xyz.b
    Traceback (most recent call last):
      File "t.py", line 9, in <module>
        import xyz.a
      File "/home/dmugtasimov/tmp/name-res3/xyz/a.py", line 11, in <module>
        import xyz.b
    ImportError: No module named b

As you see sys.path is the same for both cases:

sys.path: ['/home/dmugtasimov/tmp/name-res3', '/home/dmugtasimov/tmp/name-res3/xyz', '/usr/local/lib/python2.7/dist-packages/tornado-2.3-py2.7.egg', '/home/dmugtasimov/tmp/name-res3/xyz', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/setuptools-0.6c11-py2.7.egg-info', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

But the behaviour is different. For a.py python searches for package xyz first, and them for module b in it:

    import xyz # directory /home/dmugtasimov/tmp/name-res3/xyz
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__module.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    # /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/__init__.py
    import xyz # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/__init__.pyc
    # trying /home/dmugtasimov/tmp/name-res3/xyz/b.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/bmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/b.py
    # /home/dmugtasimov/tmp/name-res3/xyz/b.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/b.py
    import xyz.b # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/b.pyc

In other words:

    Search PACKAGE xyz in directory sys.path[0] -> FOUND
    Search module b in PACKAGE xyz -> FOUND
    Continue execution

For t.py it searches for moduel xyz in the same directory as a.py itself and then fails to find module b in module xyz:

    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyz.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyzmodule.so
    # trying /home/dmugtasimov/tmp/name-res3/xyz/xyz.py
    # /home/dmugtasimov/tmp/name-res3/xyz/xyz.pyc matches /home/dmugtasimov/tmp/name-res3/xyz/xyz.py
    import xyz.xyz # precompiled from /home/dmugtasimov/tmp/name-res3/xyz/xyz.pyc

In other words:

    Search MODULE xyz in directory in the same directory as a.py (or sys.path[1] ?) -> FOUND
    Search MODULE b in MODULE xyz -> NOT FOUND
    ImportError

So it looks like if "import xyz.b" bahaves different depending on how a.py was initially loaded as a script or imported from another module.
History
Date User Action Args
2013-01-08 08:14:31dmugtasimovsetrecipients: + dmugtasimov, docs@python
2013-01-08 08:14:31dmugtasimovsetmessageid: <1357632871.49.0.196561776247.issue16891@psf.upfronthosting.co.za>
2013-01-08 08:14:31dmugtasimovlinkissue16891 messages
2013-01-08 08:14:30dmugtasimovcreate