This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author kristjan.jonsson
Recipients jackjansen, kowaltowski, kristjan.jonsson, neologix
Date 2013-04-04.16:33:31
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1365093212.55.0.0497999086796.issue1387483@psf.upfronthosting.co.za>
In-reply-to
Content
There is no "binary" here contributing to the problem.  The particular case is a the hadoop runtime engine which creates a "virtual" folder of your working directory.  We have set up a directory of .py files:

foo/
 a.py
 b.py
 c.py

The hadoop job is then to run a.py.  It is run simply as python a.py.  In this case, by cd-ing into the dir and running the file.  hadoop knows nothing of python and merely executes the given file.

Now, what this hadoop implementation does, however, is to create a virtual symlink image of your project directory, and duplicate this in various places, e.g.:

tmp1/
 a.py -> /secret/filecache/0001/a.py
 b.py -> /secret/filecache/0002/b.py
 c.py -> /secret/filecache/0003/c.py

tmp2/
 a.py -> /secret/filecache/0001/a.py
 b.py -> /secret/filecache/0002/b.py
 c.py -> /secret/filecache/0003/c.py

Notice how each file, previously together in a folder, now pyhysically resides in a unique place.
now, when you run a.py from either tmp1 or tmp2, it will contain
sys.path[0] == "/secret/filecache/0001"

This means that a.py can no longer import b.py.  I am unaware of a workaround for this issue which does not involve modifying the individual .py files themselves to set up a path.

Fixing this would mean that rather than to
- get the absolute path of the .py file, and extract the path from it
you would
- extract the relative path of the .py file and retrieve the absolute path for it.

I am not sure about what use cases could be broken by the above change, do you have examples?
Normal use cases of symbolic links have to do with linking entire folders, not individual files, and that behaviour would not be broken by  such a hypothetical change, I think.
History
Date User Action Args
2013-04-04 16:33:32kristjan.jonssonsetrecipients: + kristjan.jonsson, jackjansen, kowaltowski, neologix
2013-04-04 16:33:32kristjan.jonssonsetmessageid: <1365093212.55.0.0497999086796.issue1387483@psf.upfronthosting.co.za>
2013-04-04 16:33:32kristjan.jonssonlinkissue1387483 messages
2013-04-04 16:33:31kristjan.jonssoncreate