This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: pkgutil doesn't understand case-senseless filesystems
Type: enhancement Stage: test needed
Components: Library (Lib) Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: fdrake Nosy List: Ankur.Ankan, Anthony.Kong, christian.heimes, eric.araujo, eric.smith, fdrake, pitrou, tarek, tim.peters
Priority: normal Keywords:

Created on 2004-04-14 18:35 by fdrake, last changed 2022-04-11 14:56 by admin.

Files
File name Uploaded Description Edit
case_ok.py tim.peters, 2004-06-19 21:26 pure-Python case_ok() function
Messages (7)
msg20509 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2004-04-14 18:35
The pkgutil.extend_path() function doesn't understand
case-senseless filesystems the way Python's import
does, but it should.  On a case-insensitive filesystem,
a directory that matches the package name in spelling
but not case can be mistakenly added to a package by
extend_path(); this can cause differently-named
packages to be mistakenly merged and allow unrelated
code to shadow code in a secondary component of a
package (where secondary means something other than the
first directory found that matches the package).

Consider this tree in a filesystem:

d1/
    foo/
        __init__.py     # this calls pkgutil.extend_path()
        module.py      # imports module "foo.something"
d2/
    Foo/
        __init__.py     # an unrelated package
        something.py
d3/
    foo/
        __init__.py
        something.py

sys.path contains d1/, d2/, d3/, in that order.

After the call to extend_path() in d1/foo/__init__.py,
foo.__path__ will contain d1/foo/, d2/Foo/, d3/foo/ (in
that order), and foo/module.py will get
d2/Foo/something.py when it imports foo.something. 
What's intended is that it get d3/foo/something.py; on
a case-sensitive filesystem, that's what would have
happened.

pkgutil.extend_path() should exercise the same care and
check the same constraints as Python's import.
msg20511 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-06-19 21:26
Logged In: YES 
user_id=31435

Attaching a pure-Python case_ok() "building block" that 
should be adequate for doing the hard part of this.  It's 
trickier than first appears because Python's case-sensitive 
imports are not sensitive to the case of extensions 
(.py, .pyo, ...) on case-insensitive filesystems.
msg20512 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2004-07-02 16:25
Logged In: YES 
user_id=3066

Assigned to me since I intend to make this work in the end.
msg59204 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2008-01-04 01:24
Fred, do you still want to fix the bug?
msg82040 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 12:54
Perhaps it's just me, but Fred's use case looks horrible.
If you have three packages named "foo", "Foo" and "foo", all of which
reachable through sys.path on a case-insensitive filesystem, I'm not
sure how to expect Python to make sense out of the situation.
IMHO, the only reasonable solution is to fix the package names...
msg82068 - (view) Author: Fred Drake (fdrake) (Python committer) Date: 2009-02-14 14:30
Antoine: I agree programmers shouldn't try to create situations like this.

Consider however an application assembled using a build tool like
zc.buildout, which installs each package into a separate installation
location (based on setuptools and easy_install).  If the application
uses several packages from the "foo" namespace (to keep with the
original names from the report), there will be directories matching the
first and third packages from the example.

Now, further development on the application may cause a 3rd-party
package named Foo to be used.  This name is not the application
programmer's to select, and the volume of code from the "foo" namespace
may be too large to modify; it may represent 3rd party code (for
example, the "zope" namespace package contains many widely used
examples; if used at all, many are likely to be used).

So I think the original use case stands, and may even be more important
than ever given the widespread use of "namespace" packages.

This also suggests that developers should check for the existence of
packages of similar name before releasing code, in order to avoid this
situation.

(I'm disassociating msg20510, since roundup doesn't screw up the tree in
the original report, so that message no longer adds anything useful.)
msg185584 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-03-30 19:25
With the advent of implicit namespace packages in 3.3, I don’t know if anything in pkgutil will be done for this use case.

(BTW is it normal that pkgutil.extend_path is not at least doc-deprecated?)
History
Date User Action Args
2022-04-11 14:56:03adminsetgithub: 40152
2013-06-19 18:46:44Ankur.Ankansetnosy: + Ankur.Ankan
2013-03-30 19:25:09eric.araujosetnosy: + eric.araujo, eric.smith
messages: + msg185584
2013-03-29 03:01:09Anthony.Kongsetnosy: + Anthony.Kong
2011-03-09 03:23:40terry.reedysetnosy: tim.peters, fdrake, pitrou, christian.heimes, tarek
versions: + Python 3.3, - Python 3.1, Python 2.7, Python 3.2
2010-11-06 21:39:41eric.araujosetversions: + Python 3.1, Python 3.2
2009-02-14 14:31:14fdrakesetmessages: - msg20510
2009-02-14 14:31:00fdrakesetmessages: + msg82068
2009-02-14 12:54:51pitrousetnosy: + pitrou
messages: + msg82040
2009-02-14 12:31:36ajaksu2setnosy: + tarek
stage: test needed
versions: + Python 2.7, - Python 2.6, Python 2.5
2008-01-04 01:24:01christian.heimessetnosy: + christian.heimes
type: enhancement
messages: + msg59204
versions: + Python 2.6, Python 2.5, - Python 2.3
2004-04-14 18:35:19fdrakecreate