On several occasions I have seen tracebacks in my code
pointing to PIL's __init__.py file. That is strange,
because I have installed PIL but it is used nowhere.

Finally I traced it down to a problem in the linecache
code, which tries to be smart in up updatecache
function.  When os.stat() on the filename fails with
os.error, it walks along sys.path and returns the first
file with a matching basename. This *may* make sense
for toplevel modules, but never for modules in packages.

So, if the traceback's stack contains an entry for a
non-existing file (for example because the .py file for
a .pyc file is no longer present), linecache returns
absolute garbage.

Example, on my system (the \a\b\c\__init__.py file
doesn't exist):

C:\>python -c "import linecache; print
linecache.getlines(r'\a\b\c\__init__.py')"
['#\n', '# The Python Imaging Library.\n',
'# $Id: //modules/pil/PIL/__init__.py#2 $\n', '#\n',
'# package placeholder\n', '#\n', '# Copyright (c) 1999
by Secret Labs AB.\n',
'#\n', '# See the README file for information on usage
and redistribution.\n',
'#\n', '\n', '# ;-)\n']
C:\>

The bug is present in 2.3, 2.4, and current CVS.

Logged In: YES 
user_id=11105

The attached patch is a radical way to fix this problem - it
refuses to guess if the file is not found.

Logged In: YES 
user_id=6380

Radical indeed. But what about the use case of modules
compiled on one system and moved to a different system? Is
that use case important enough?

There used to be a different use case, because bytecode
files used to contain relative path names. I believe that's
been fixed, if only by code in site.py that absolutizes
sys.path. (Except the initial '' -- is that important?)

Logged In: YES 
user_id=11105

> But what about the use case of modules compiled on one system
> and moved to a different system? Is that use case important
> enough?

Maybe.  Couldn't the code be made more robust, so that it
works as intended with packages?  Unfortunately the code
objects don't have a reference to the module where they live
(probably because they are created before the module is
created).

BTW: The bug sometimes also is responsible for tracebacks in
pydoc, because it cannot locate the sourceline in the file.
 So, wrong sourec in tracebacks aren't the only problem.

It seems it would work better with relative pathnames, but
it's too late for that.

The following issues appear to be the same bug to me:

   http://bugs.python.org/issue1068477
   http://bugs.python.org/issue1309567
   http://bugs.python.org/issue1754483

and are, as of 2008-06-30, unfixed in both rel25-maint and trunk.

Is this still relevant?

Unless someone has fixed it since 2008-07-05, it is still relevant.

I'll have to take a look at the current code first, though, to confirm
in either way.

There has been some discussion on this issue over at
http://bugs.python.org/issue1754483 but no complete resolution either.

This is still relevant.  It makes linecache (and therefore some other
things, like inspect.getsource) almost useless in many cases.  For
example, because of this, sometimes inspect.getsource will raise an
IOError and sometimes it will return garbage data.  If the source file
isn't around, it'd be much better to fail all the time.  Rummaging
around in the filesystem hoping to get lucky and find another file with
the same name which contains the correct contents is insane.  This may
have made sense when relative path names were being used, but it makes
no sense anymore.

This seems related to http://bugs.python.org/issue1218234 as well. 
This is my first bug report. I hope I do this right.
I posted this to comp.lang.python and was directed here. Here is a
chopped paste of that post:

I am getting an error on line 510 of inspect.py:

504    if iscode(object):
505        if not hasattr(object, 'co_firstlineno'):
506            raise IOError('could not find function definition')
507        lnum = object.co_firstlineno - 1
508        pat = re.compile(r'^(\s*def\s)|(.*(?<!\w)lambda(:|\s))|^(\s*@)')
509        while lnum > 0:
510            if pat.match(lines[lnum]): break
511            lnum = lnum - 1
512        return lines, lnum

I finally figured out that there was a caching problem. The function I
passed was changed, but the code lines (strings) retrieved by
linecache.getlines() (on lines 464 and 466) didn't update with the new
module contents... To get around this, I invoke linecache.clearcache()...

    linecache.clearcache()
    lines, n = inspect.getsourcelines(fn)


While inspect uses the cached module to get the text, the line number
used to find the block of source lines is retrieved from the passed
object. So, if I pass a function which reports that it starts on line
50, but in the cache it starts on line 40, an error isn't raised but the
lines of code returned are wrong. The error only occurs when the line
number is higher than the number of lines in the cached module.

Cheers,

- Rafe

I just had the same problem in a call to inspect.findsource() 
(actually my code calls inspect.stack() which calls ... which calls 
findsource()). The code object passed to findsource() had a filename 
of "database/__init__.py" which is the correct filename relative to 
sys.path, so in this case a simple fix would have worked: delete line 
82 of linecache.py
 82        basename = os.path.split(filename)[1]
and use filename instead of basename for the rest of the function. (Or 
for a quick hack, change line 82 to just "basename = filename")

I'm using Python 2.5.2 but the relevant code in linecache.py also 
exists in 3.0 (just checked).

Adding some tests to make working on a patch easier.

Patch (against py3k) with David Zaslavsky's solution from msg79117. It
would be good to find failing tests after it's applied so we can fix
this without breaking valid use cases.

Applied patch and added test suite in r72319.