Author ncoghlan
Recipients eryksun, ncoghlan, ned.deily, nedbat, steve.dower
Date 2017-03-08.07:45:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1488959123.98.0.225674963564.issue29723@psf.upfronthosting.co.za>
In-reply-to
Content
Ah, interesting, I didn't know there was a difference between the platforms in when the placeholder got resolved to a full path.

However, after browsing the code and running some local tests, it seems that injecting sys.path[0] isn't handled by Py_GetPath() or PySys_SetPath(), regardless of OS.

Instead, it's handled as a side effect of calling PySys_SetArgV(), with PySys_SetArgVEx() adding a flag to disable the side effect (for the benefit of the isolated mode implementation).

The actual side effect itself is implemented in sys_update_path: https://github.com/python/cpython/blob/3.6/Python/sysmodule.c#L2162

So in the case of running `./python Tools` (which exercises the paths of interest here) with some strategically placed printf() and PySys_FormatStdout() calls, I get:

```
$ ./python Tools
module_search_path: /usr/local/lib/python36.zip:/home/ncoghlan/devel/py36/Lib:/home/ncoghlan/devel/py36/Lib:/home/ncoghlan/devel/py36/build/lib.linux-x86_64-3.6
sys.path[0]: '/home/ncoghlan/devel/py36'
sys.path: ['/home/ncoghlan/devel/py36', '/usr/local/lib/python36.zip', '/home/ncoghlan/devel/py36/Lib', '/home/ncoghlan/devel/py36/build/lib.linux-x86_64-3.6', '/home/ncoghlan/.local/lib/python3.6/site-packages']
/home/ncoghlan/devel/py36/python: can't find '__main__' module in 'Tools'
```

The first line is from `Py_GetPath()`, the second is from `sys_update_path()`, the third is from `RunMainFromImporter` (before it makes any sys.path changes), and the last is the expected error because our `Tools` directory isn't executable.

In this scenario, we want the "Tools" entry to *overwrite* sys.path[0].

While in isolated mode I get:

```
$ ./python -I Tools
module_search_path: /usr/local/lib/python36.zip:/home/ncoghlan/devel/py36/Lib:/home/ncoghlan/devel/py36/Lib:/home/ncoghlan/devel/py36/build/lib.linux-x86_64-3.6
sys.path: ['/usr/local/lib/python36.zip', '/home/ncoghlan/devel/py36/Lib', '/home/ncoghlan/devel/py36/build/lib.linux-x86_64-3.6']
/home/ncoghlan/devel/py36/python: can't find '__main__' module in 'Tools'
```

In this scenario, we want the "Tools" entry to be inserted *before* sys.path[0].

Note that this has been buggy since the -I switch was introduced, but the entry we've been overwriting has been the one for the stdlib-as-a-zip-archive, so it didn't cause any problems for anyone using the default directory-based installation layout.

However, we can't reliably figure out which to do based on the contents of sys.path, we need to look at Py_IsolatedFlag instead. (I'm not sure why the current check is working on Windows, as sys_update_path() attempts to resolve an absolute path entry from the archive or directory name there as well)
History
Date User Action Args
2017-03-08 07:45:24ncoghlansetrecipients: + ncoghlan, nedbat, ned.deily, eryksun, steve.dower
2017-03-08 07:45:23ncoghlansetmessageid: <1488959123.98.0.225674963564.issue29723@psf.upfronthosting.co.za>
2017-03-08 07:45:23ncoghlanlinkissue29723 messages
2017-03-08 07:45:23ncoghlancreate