setuptools 0.6c10 is released with fixes for this issue.

It appears that a big source of spurious warnings for this is when pkg_resources
is imported after other modules found in eggs.  This can be resolved by changing
the insert_on() method to only check conflicts when the distribution isn't
already on sys.path.  In other words, if you're re-adding something that's
already there, there's no point in getting the warning more than once.  In
addition, wonlay's patch should help to fix some of the remaining cases.

That leaves handling of packages.  It appears that the issue there is that the
setup.py for 'foolscap' is broken: its 'packages' keyword contains *paths*
rather than package names, which causes setuptools to generate a hosed
'top-level.txt', which should only contains top level package names.  This is
causing attempts to check consistency of these nested packages, which then
breaks with stow.  I'll add a warning for this, so that other people won't make
this mistake with their packages in future.

ok, here's a patch which resolves my particular problem, not sure if it's the
right thing to do or not:

diff --git a/pkg_resources.py b/pkg_resources.py
index 9edb6c0..b812371 100644
--- a/pkg_resources.py
+++ b/pkg_resources.py
@@ -2267,8 +2267,15 @@ class Distribution(object):
                 continue
 
             fn = getattr(sys.modules[modname], '__file__', None)
-            if fn and normalize_path(fn).startswith(loc):
+            if fn and (fn.startswith(loc) or normalize_path(fn).startswith(loc)):
                 continue
+            pieces = fn.split(os.path.sep)
+            if (len(pieces) > 2
+                and pieces[-1] in ("__init__.py", "__init__.pyc")
+                and pieces[-2] == modname):
+                fn2 = os.path.sep.join(pieces[:-2])
+                if normalize_path(fn2).startswith(loc):
+                    continue
             issue_warning(
                 "Module %s was already imported from %s, but %s is being added"
                 " to sys.path" % (modname, fn, self.location),

wonlay: thanks for the great hint! I've been seeing a similar problem at
home, on my OS-X box, and thanks to your suggestion I was able to figure out
the problem. I don't yet know the solution, though :).

My general policy is to only ever to 'sudo install' when the install target
is to a stow-managed per-package-per-version directory. So in this case, when
I installed the "foolscap" package, I put it in
/usr/local/stow/foolscap-0.4.2/ , and then told stow to symlink it into place
at /usr/local/lib/python2.5/site-packages .

I then see this warning any time anything imports foolscap.

When pkg_resources.Distribution.check_version_conflict() is asked (for some
reason, I don't know why) to check on the 'foolscap' package, it sees that
the __file__ attribute points to:

 /usr/local/lib/python2.5/site-packages/foolscap/__init__.pyc

When it does a normalize_path() on that, it is told that __init__.pyc lives
in:

 /usr/local/stow/foolscap-0.4.2/lib/python2.5/site-packages/foolscap/__init__.pyc

and indeed that is not on sys.path, so it emits the warning.

I guess what I want it to do is to figure out where the module really got
imported from first: chop off the module name and the __init__.pyc to get
"/usr/local/lib/python2.5/site-packages", then do a normalize_path() on that,
and *then* compare it against 'loc'. I don't know how to do that offhand..
will there always be an __init__.pyc file?

cheers,
 -Brian

In my case importing any systemwide package installed causes this warning:

jarek:~/dev/cruncher$ PYTHONPATH=$HOME/dev/cruncher/cruncher/lib python
>>> import cruncher
>>> import pkg_resources
__main__:1: UserWarning: Module cruncher was already imported from 
cruncher/__init__.pyc, but /Users/jarek/dev/cruncher/cruncher/lib is being added 
to sys.path
>>> 
jarek:~/dev/cruncher$ PYTHONPATH=$HOME/dev/cruncher/cruncher/lib python
>>> import cruncher
>>> import psycopg2
/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-
packages/psycopg2-2.0.11-py2.5-macosx-10.5-i386.egg/psycopg2/_psycopg.py:3: 
UserWarning: Module cruncher was already imported from cruncher/__init__.pyc, 
but /Users/jarek/dev/cruncher/cruncher/lib is being added to sys.path
  import sys, pkg_resources, imp
>>>

Yes, wonlay's fix (around line 2270) works fine for me.

On my Ubuntu machine, a quick way to show the bug is:

>>> import MySQLdb
>>> import pkg_resources

__main__:1: UserWarning: Module _mysql was already imported from 
/var/lib/python-support/python2.5/_mysql.so, but /var/lib/python-
support/python2.5 is being added to sys.path

Okay, I've learned that one of the instances of this warning was a bug in
nevow's setup.py that was declaring that nevow came with a package named
twisted.plugins:

http://divmod.org/trac/ticket/2830

It sounds like wonlay is onto a bigger issue though.

Sorry missed some words last submit. It's:

After debuging, I found the warning message is from function
check_version_conflict in pkg_resources.py

I'm facing this issue now. In my case, the module showed in already import
warning is _mysql.
You can google for:
http://www.google.com/search?hl=en&newwindow=1&safe=off&rlz=1B3GGGL_en___CN232&q=UserWarning%3A+Module+was+already+imported+from+but+%2Fvar%2Flib%2Fpython-support%2Fpython2.5+is+being+added+to+sys.path&btnG=Search

Lots of people have the same problem, and several bug have been reported to
several projects, but it's hard to reproduce, because this is not there issue,
it's issue of pkg_resources in setuptools I think.

I don't know how to reproduce this bug simply, but you can reproduce it follow
these steps (I see this bug when I'm using Django and Protobuf together):

(I'm using debian lenny, python 2.5)
1. Install mysql and its python binding MySQLdb;
2. Install django use mysql as your db backend(http://www.djangoproject.com/);
3. Install protobuf (Both the library and python bindings inside the download
package.)(http://code.google.com/apis/protocolbuffers/);
4. Start a your django project, start a new app, add your app models;
5. Create a simple .proto protobuf file and compile it into .py;
6. Import the compiled model in the end of your django app's models.py;
7. Run ./manage.py shell;

You will see the warning message:
/usr/lib/python2.5/site-packages/protobuf-2.0.2-py2.5.egg/google/__init__.py:1:
UserWarning: Module _mysql was already imported from
/var/lib/python-support/python2.5/_mysql.so, but
/var/lib/python-support/python2.5 is being added to sys.path

After debuging, I found the warning message is from function 

when modname is _mysql, after:
fn = getattr(sys.modules[modname], '__file__', None)

fn will be '/var/lib/python-support/python2.5/_mysql.so'.

But normalize_path(fn) is
'/usr/lib/python-support/python-mysqldb/python2.5/_mysql.so'.

So normalize_path(fn).startswith(loc) failed(loc is
'/var/lib/python-support/python2.5').

Actually, '/var/lib/python-support/python2.5/_mysql.so' is a symbolic link of
'/usr/lib/python-support/python-mysqldb/python2.5/_mysql.so'.

My workaround is first test fn.startswith(loc) and then
normalize_path(fn).startswith(loc).

And the patch attached.

There is a discussion about this issue on the tahoe-dev list:

http://allmydata.org/pipermail/tahoe-dev/2008-September/000785.html

So it seems increasingly likely that this is yet another problem caused by Nevow
importing "nevow" and "twisted" when its setup.py is executed: please see issue20

These steps do not reproduce the problem.  I built the Nevow egg using
"easy_install Twisted Nevow".  The subdirectory I used was 't' rather than
'FOO', but I don't think that's relevant.  For that matter, I don't see how the
given steps could possibly produce the error described, unless zope.interface
was doing a require('Nevow'), and importing twisted caused that to happen.

Sorry about that -- I didn't examine the log that I was posting to realize that
it didn't echo the actual Python command-line.

Sorry about that, I should have investigated this problem further before raising
it to zooko.

To reproduce this problem independently:
 1: create Nevow-0.9.31-py2.5.egg
 2: put it in a child directory FOO
 3: run PYTHONPATH=FOO python -c 'import twisted'

I haven't yet been able to figure out how to accomplish step 1 outside of our
Tahoe build system. We do it with a "setup.py develop" that sees a dependency on
Nevow that needs satisfying. Apparently there is something broken in the Nevow
setup.py (or rather their setup_egg.py), as a regular "bdist_egg" doesn't seem
to work.

Please stop submitting "make" logs as examples; they are useless to me and
anyone else who wants to work on the bug and doesn't have your makefiles,
doesn't run on the same platform, etc.  Thanks.

/usr/lib/python2.5/site-packages/zope/__init__.py:19: UserWarning: Module
twisted was already imported from
/usr/lib/python2.5/site-packages/twisted/__init__.pyc, but
/home/buildslave/tahoe/feisty2.5/build/support/lib/python2.5/site-packages/Nevow-0.9.31-py2.5.egg
is being added to sys.path

Here is an example command line that yields this warning:

http://allmydata.org/buildbot/builders/feisty2.5/builds/1649/steps/pyflakes/logs/stdio

Note that this may be related to issue17 and issue20 -- why is the Nevow egg
declaring the "twisted" namespace and/or module anyway?