classification
Title: "python.exe t2.py" doesn't work the same on Python-3.6 as Python-3.5
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Big Stone, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2017-02-16 08:56 by Big Stone, last changed 2017-02-16 19:57 by Big Stone. This issue is now closed.

Messages (21)
msg287919 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 08:56
Hi,

I have two file "t1.py" and "t2.py" in the same directory. "t2.py" can't import "t1.py" when launched from python.exe on Python-3.6. it was ok on Python-3.5 (all other things being equal). And it's ok when launched from IDLE.

Has there been a volontary change ?
t1.py:
print("i am t1")

t2.py:
import t1
print("I am t2")

result (on WinPython-3.6):
python.Exe C:\WinPython\basedir36\buildZero\winpython-64bit-3.6.x.2\test\t2.py

Traceback (most recent call last):
  File "C:\WinPython\basedir36\buildZero\winpython-64bit-3.6.x.2\test\t2.py", line 1, in <module>
    import t1
ModuleNotFoundError: No module named 't1'

C:\WinPython\basedir36\buildZero\winpython-64bit-3.6.x.2\scripts>

nota: Winpython specificity is the use of python._pth, with:
python36.zip
DLLs
Lib
.
import site
msg287920 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 09:07
This sounds like a bug in winpython, not in Python itself. You need the location of t1.py in your _pth file. See https://docs.python.org/3.6/using/windows.html#finding-modules for details.

Python 3.5 didn't use the _pth file mechanism, which is why the behaviour is different.
msg287921 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 09:39
doesn't the "." entry means "look at same place as current source file" ? Could there be a way to get that "classic" behavior on Python-3.6 ?
msg287923 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 09:46
No (see the doc link I referenced) - paths are absolute, or relative to the _pth file. So "." means "in the same place as the pth file".

I don't think there's a way with _pth files to get the "add the location of the executed script to the front of sys.path" behaviour. It's not really a good idea for an embedded interpreter (which is the _pth file intended use case) as it makes it a bit too easy to run code from unexpected locations.

In an embedded application, you could of course add sys.path entries in your C code. Maybe WinPython could do that too?
msg287927 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 10:06
The "targeted" user, a beginner in Python, will put its t1.py & t2.py examples anywhere on his disk, and discover that it doesn't work like in the book.

It is very annoying for WinPython intended purpose, if a simple trick cannot provide the beginner "expected" behavior.
msg287928 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 10:13
and I fail to understand why IDLE doesn't feel the problem.
msg287932 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 11:53
ok, I found this as a possible workaound. I hope it's correct

on t2.py, specify the __path__ variable before importing t1.

import os
__path__=[os.path.dirname(os.path.abspath(__file__))]
from . import t1
print("t2 done")
msg287935 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 12:29
I'm not sure about this, I've never seen __path__ used like this. Why can't you just set sys.path?

    sys.path.append(os.path.dirname(os.path.abspath(__file__)))

That's how scripts typically adjust their search path if needed.
msg287936 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 13:29
Using sys.path.append could result in an arbitrary long sys.path, full of duplicates, isn't it ?
msg287939 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 13:38
No more so than any other method of adding entries to sys.path (which is what __path__ does for packages, I've just never seen it used for modules).
msg287940 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-16 13:39
The ._pth file is certainly not meant to cover beginner scenarios. But with that list of paths you should get almost exactly the same sys.path by default.

Modifying a module's __path__ will only affect that module (and imports via that module), whereas sys.path affects your entire program. Adding an empty string to either should give you the default behavior back, but this is specifically not supported by ._pth.
msg287947 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 14:07
Hi Steve,

Could there be a "<blank>" (or "@source" or whatever string) convention be added in python._pth to explicitely allow that "formerly default" situation ? 

It would not break anything backward, and allow again classic Python-3.5 code to work unchanged in all circumstances (of WinPython) again.



in python-3.6.1 ?
msg287948 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 14:10
or just I add a blank line in current python._pth and all is ok immediately ?
msg287949 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 14:22
If I place a blank line or a semi-column, nothing happen.
If i put my relative path ..\test, then I must do "import t1"
If I set a __path__ in t2.py, then I have to do "from . import t1"

"There should be one-- and preferably only one --obvious way to do it.", but I don't clearly see it.
msg287952 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-16 15:51
The semantics of the ._pth file won't be changing to accommodate security vulnerabilities, sorry. Add a sitecustomize.py file to modify sys.path if you want that behavior.
msg287954 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 16:04
It's probably worth also saying that maybe winpython shouldn't even be using the _pth file feature. I don't know why it is, but the intended use case for _pth files is embedded systems, so it's not clear how an alternative standalone Python interpreter matches that scenario...
msg287955 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-16 16:07
> an alternative standalone Python interpreter

It's a convenient way to avoid having your standard library hijacked by registry keys installed by the regular interpreter.

However, if it detects "Lib\os.py" or "python36.zip" alongside the executable, it shouldn't look in the registry to figure out its home directory. That ought to be sufficient for portable cases, though of course there are some registry entries that will still have an impact (until 3.6.1, when we should be registry clean provided one of the landmark files is found).
msg287957 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-02-16 16:24
>> an alternative standalone Python interpreter
>
> It's a convenient way to avoid having your standard library hijacked by registry keys installed by the regular interpreter.

Ah yes, that makes sense - it's maybe not the *right* way, but as a
practical approach it has benefits :-)

> However, if it detects "Lib\os.py" or "python36.zip" alongside the executable, it shouldn't look in the registry to figure out its home directory. That ought to be sufficient for portable cases, though of course there are some registry entries that will still have an impact (until 3.6.1, when we should be registry clean provided one of the landmark files is found).

And that of course is the "right" way - it's just taken a long time to
get there because of all the nastiness involved in interpreter startup
(see PEP 432 :-))

I'd forgotten (and hadn't gone and checked) that WinPython was
designed as a portable build.
msg287958 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 17:12
ok, I understand that the improvement over current solution is, starting with python-3.6.1rc:
- remove "python._pth"
- check I have a "Lib\os.py" (otherwise a "python36.zip"),
- and no pixie dust here or there ?
msg287960 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2017-02-16 17:32
> starting with python-3.6.1rc

Works with 3.6.0. The only relevant changes in 3.6.1 are to skip empty lines in the ._pth file, and to avoid overwriting sys.path[0] arbitrarily.
msg287970 - (view) Author: Big Stone (Big Stone) Date: 2017-02-16 19:57
Thanks Steve.

Change applied, byebye python._pth.
History
Date User Action Args
2017-02-16 19:57:16Big Stonesetmessages: + msg287970
2017-02-16 17:32:03steve.dowersetmessages: + msg287960
2017-02-16 17:12:07Big Stonesetmessages: + msg287958
2017-02-16 16:24:35paul.mooresetmessages: + msg287957
2017-02-16 16:07:42steve.dowersetmessages: + msg287955
2017-02-16 16:04:18paul.mooresetmessages: + msg287954
2017-02-16 15:51:31steve.dowersetmessages: + msg287952
2017-02-16 14:22:14Big Stonesetmessages: + msg287949
2017-02-16 14:10:47Big Stonesetmessages: + msg287948
2017-02-16 14:07:13Big Stonesetmessages: + msg287947
2017-02-16 13:39:09steve.dowersetmessages: + msg287940
2017-02-16 13:38:42paul.mooresetmessages: + msg287939
2017-02-16 13:29:12Big Stonesetmessages: + msg287936
2017-02-16 12:29:49paul.mooresetmessages: + msg287935
2017-02-16 11:53:20Big Stonesetmessages: + msg287932
2017-02-16 10:13:08Big Stonesetmessages: + msg287928
2017-02-16 10:06:25Big Stonesetmessages: + msg287927
2017-02-16 09:46:50paul.mooresetmessages: + msg287923
2017-02-16 09:39:30Big Stonesetmessages: + msg287921
2017-02-16 09:07:03paul.mooresetstatus: open -> closed
resolution: not a bug
messages: + msg287920

stage: resolved
2017-02-16 08:56:28Big Stonesettitle: "python.exe t2.py" doesn't work the same on Pythn-3.6 as Python-3.5 -> "python.exe t2.py" doesn't work the same on Python-3.6 as Python-3.5
2017-02-16 08:56:15Big Stonecreate