classification
Title: Compiler generates relative filenames
Type: Stage:
Components: Interpreter Core Versions:
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Ramchandra Apte, aronacher, brett.cannon, eric.araujo, eric.snow, glchapman, gvanrossum, isandler, jhylton, ncoghlan, prescod
Priority: normal Keywords:

Created on 2001-04-11 18:52 by prescod, last changed 2012-06-24 15:44 by brett.cannon. This issue is now closed.

Messages (11)
msg4213 - (view) Author: Paul Prescod (prescod) * (Python triager) Date: 2001-04-11 18:52
Make a file called "test.py"
----
import pack

print pack.func.func_code.co_filename
-----

Make a directory called "pack". Put a file in it 
called "__init__.py" with the contents:

def func(): pass

Now run test.py. It will compile a relative path into 
these modules. Now you can change to any directory on 
the system and run test.py and it will return the 
original relative path. The problem is that the 
relative path is compiled into the .pyc. It should be 
an absolute path.
msg4214 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2001-04-13 14:12
Logged In: YES 
user_id=31392

I'm not clear on what the rules for co_filename are, but it
looks like the absolute path is only used for package
imports.  If you cd into the package directory, run python
-c "import __init__", and then later import the package the
path name is relative.

The compiler package isn't connected to the import
mechanism, so there's no way for it to know whether it is
compiling a module or a package.  I don't think there's a
way to do anything better.
msg4215 - (view) Author: Jeremy Hylton (jhylton) (Python triager) Date: 2001-04-13 21:15
Logged In: YES 
user_id=31392

Just to clarify.  The compiler in question is the builtin
compiler.  It generates absolute path names unless the .py
file is in the current working directory.
msg4216 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2001-09-05 18:10
Logged In: YES 
user_id=6380

Why on earth was this assigned to Paul? He's not going to
make progress. Assigning back to nobody.
msg4217 - (view) Author: Greg Chapman (glchapman) Date: 2002-04-25 21:08
Logged In: YES 
user_id=86307

I just ran into this today when trying to find out why 
pydoc wasn't working for a module of mine: 
inspect.getmodule calls os.abspath with the filename found 
in the code object of global functions.  If the cwd is 
different than when the code object was compiled, 
inspect.getmodule fails.

Anyway, it looks to me like most of these relative paths 
could be caught in the find_module function (of import.c) 
if, when given an empty string for a path (while iterating 
over sys.path), find_module called getcwd() and used that 
instead of the empty string.  Of course, this assumes that 
(on all platforms) opening a bare filename means open the 
file with that name in the current directory (is that a 
valid assumption?).  Also, it appears from posixmodule.c 
that getcwd may not always be available.

Does this sound like a reasonable idea?
msg4218 - (view) Author: Ilya Sandler (isandler) Date: 2005-04-23 04:02
Logged In: YES 
user_id=971153

I tried to reproduce the problem with python 2.4 and I'm
getting the absolute path....

So, I guess, the bug can be closed
msg4219 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2005-04-26 21:54
Logged In: YES 
user_id=593130

Closing as fixed, on the basis of isandlers report, until someone 
verifies that it is a current problem.
msg145650 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2011-10-16 23:01
It's fairly easy to check this is still a problem:

$ ./python
Python 3.3.0a0 (default:a06ef7ab7321, Sep 22 2011, 13:41:29) 
[GCC 4.6.0 20110603 (Red Hat 4.6.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setup
>>> setup.__file__
'setup.py'

That's a relative path rather than an absolute one. If you edit sys.path to use '.' instead of '', it becomes clear that the import machinery is just doing a blind join of the sys.path entry with the relative location of the file:

>>> del sys.modules['setup']
>>> sys.path[0] = '.'
>>> import setup
>>> setup.__file__
'./setup.py'
msg145652 - (view) Author: Armin Ronacher (aronacher) * (Python committer) Date: 2011-10-16 23:32
The reason why this is a problem:

$ cat test.py
def foo():
    pass


>>> import test, os, inspect
>>> os.chdir('/')
>>> inspect.getsource(test)
'def foo():\n    pass\n'

But

>>> import test, os, inspect
>>> os.chdir('/')
>>> inspect.getsource(test)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: source code not available
msg163250 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2012-06-20 05:42
What is the status of this bug?
This is the oldest open bug.
msg163803 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-06-24 15:44
So co_filename is absolute in Python 3.3, but __file__ on modules is not if a relative path is used from sys.path (as Nick pointed out). Changing this would possibly break code as this has been baked into the import system for quite some time. I think Nick as suggested making all paths absolute for __file__, but I can't remember where that discussion was held and why we never changed the semantics.

Regardless, __file__ being absolute is a different issue than co_filename, so I'm closing this at out of date.
History
Date User Action Args
2012-06-24 15:44:12brett.cannonsetstatus: open -> closed
resolution: out of date
messages: + msg163803
2012-06-20 05:42:43Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg163250
2011-10-21 21:46:21eric.araujosetnosy: + eric.araujo
2011-10-17 14:04:56Arfreversetnosy: + Arfrever
2011-10-17 02:39:33terry.reedysetnosy: - terry.reedy
2011-10-17 02:15:24eric.snowsetnosy: + eric.snow
2011-10-16 23:32:32aronachersetnosy: + aronacher
messages: + msg145652
2011-10-16 23:01:04ncoghlansetstatus: closed -> open

nosy: + brett.cannon, ncoghlan
messages: + msg145650

resolution: fixed -> (no value)
2001-04-11 18:52:39prescodcreate