This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Wrong virtual environment found
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: matrixise, paul.moore, sfx2k, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2018-08-08 14:30 by sfx2k, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (9)
msg323278 - (view) Author: sfx2k (sfx2k) Date: 2018-08-08 14:30
For my python development I have several directories on my Windows system:

common:
F:\python\scripts -> one-file-scripts based on python 2
F:\python\scripts3 -> one-file-scripts base on python 3

projects:
F:\python\projects\timetracking
...

Each directory has its own virtual environment (managed with pipenv):
F:\python\scripts\.venv
F:\python\scripts3\.venv
F:\python\projects\timetracking\.venv

Because I want to be able to call the scripts from everywhere, I added the directories to the path-environment variable.

[...]F:\Python\scripts3;F:\Python\projects\timetracking[...]

Let's have a look at the timetracking project. The main script (timetracking.py) has the following shebang-line:
#! /usr/bin/env .venv/scripts/python.exe

My current directory is the root of C

When I call 'timetracking.py' I get an error that some modules have not been installed.

With activating the debug mode (set PYLAUNCH_DEBUG=1) I get some more information:

***
C:\>timetracking.py
launcher build: 32bit
launcher executable: Console
File 'C:\Users\itsme\AppData\Local\py.ini' non-existent
File 'C:\Windows\py.ini' non-existent
Called with command line: "F:\Python\projects\timetracking\timetracking.py"
maybe_handle_shebang: read 256 bytes
maybe_handle_shebang: BOM not found, using UTF-8
parse_shebang: found command: F:\Python\scripts3\.venv\scripts\python.exe
run_child: about to run 'F:\Python\scripts3\.venv\scripts\python.exe "F:\Python\projects\timetracking\timetracking.py" '
Traceback (most recent call last):
  File "F:\Python\projects\timetracking\timetracking.py", line 18, in <module>
    import menu_definitions
  File "F:\Python\projects\timetracking\menu_definitions.py", line 15, in <module>
    import menu_edit_item_functions
  File "F:\Python\projects\timetracking\menu_edit_item_functions.py", line 15, in <module>
    import tools
  File "F:\Python\projects\timetracking\tools.py", line 19, in <module>
    from texttable import Texttable
ModuleNotFoundError: No module named 'texttable'
child process exit code: 1
***

As you can see, the pylauncher found a shebang - but not that one I expected ('scripts3\.venv' instead of 'timetracking\.venv') *confused*
It seems that the pylauncher uses the first .venv it found within the path variable...

Any ideas?
msg323279 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-08-08 14:40
From https://docs.python.org/3.7/using/windows.html#shebang-lines the supported shebang lines are

* /usr/bin/env python
* /usr/bin/python
* /usr/local/bin/python
* python

There's a provision in there:

"""
The /usr/bin/env form of shebang line has one further special property. Before looking for installed Python interpreters, this form will search the executable PATH for a Python executable. This corresponds to the behaviour of the Unix env program, which performs a PATH search.
"""

The launcher does *not* implement the full functionality of the Unix "env" program, and in particular doesn't support (relative or absolute) paths in the Python interpreter part.

While "search PATH for the relative path given in the env shebang line" would be a potential feature request, it's not current behaviour, and in my view, it's too rare of a scenario to be worth the complexity it would add to the launcher.

As a workaround, you can use an absolute path in your shebang line:

#!F:\python\scripts3\.venv\Scripts\python.exe

(Obviously that requires a little more manual management of the shebang lines in your scripts).
msg323280 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-08-08 14:50
And the Shebang is specific to the Unix-like systems, the loader from the operating system, will read the first line and will try to execute the defined interpreter. 

https://en.wikipedia.org/wiki/Shebang_(Unix)

On Windows, I am not sure, but like Paul told you, PyLauncher does not implement the full functionality of the Unix "env" program.

Maybe we could close this issue.
msg323281 - (view) Author: sfx2k (sfx2k) Date: 2018-08-08 14:54
Thanks for your feedback, Paul and Stephane.

The thing with the relative path works if I change the order in the path from [...]F:\Python\scripts3;F:\Python\projects\timetracking[...] to [...]F:\Python\projects\timetracking;F:\Python\scripts3[...] 

;)

An absolut path is not practicable because it can be different on other systems :)

How do others do it? I don't think that I am the only one that uses such a workflow, or am I wrong?
msg323282 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-08-08 14:55
Others use absolute paths, as I mentioned in my reply.
msg323283 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2018-08-08 14:57
Oh, and I suspect that the reason that "The thing with the relative path works if I change the order in the path" is simply because the bit of code that searches PATH just ignores what's after #!/usr/bin/env and looks for python.exe. (More or less - I haven't checked the code).
msg323284 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-08-08 14:59
Now, you could use a Windows shortcut where you specify the version of Python when you execute your script.

F:\python\scripts3\.venv\bin\python myscript.py

For example.
msg323287 - (view) Author: sfx2k (sfx2k) Date: 2018-08-08 15:07
Okay, thanks anyway :)

btw:

"Oh, and I suspect that the reason that "The thing with the relative path works if I change the order in the path" is simply because the bit of code that searches PATH just ignores what's after #!/usr/bin/env and looks for python.exe. (More or less - I haven't checked the code)."

Nope - then it would find the python executable in c:\python3 first ;)
msg323289 - (view) Author: Stéphane Wirtel (matrixise) * (Python committer) Date: 2018-08-08 15:09
it's normal because this path is in the PATH env variable.

so in this case, I think we can close this issue.
History
Date User Action Args
2022-04-11 14:59:04adminsetgithub: 78540
2018-08-08 15:11:13sfx2ksetstatus: open -> closed
stage: resolved
2018-08-08 15:09:27matrixisesetmessages: + msg323289
2018-08-08 15:07:19sfx2ksetmessages: + msg323287
2018-08-08 14:59:15matrixisesetstatus: closed -> open

nosy: + matrixise
messages: + msg323284

resolution: not a bug ->
stage: resolved -> (no value)
2018-08-08 14:57:38paul.mooresetmessages: + msg323283
2018-08-08 14:55:48paul.mooresetmessages: + msg323282
2018-08-08 14:55:11paul.mooresetstatus: open -> closed
resolution: not a bug
stage: resolved
2018-08-08 14:54:49sfx2ksetnosy: - matrixise
messages: + msg323281
2018-08-08 14:50:24matrixisesetnosy: + matrixise
messages: + msg323280
2018-08-08 14:40:36paul.mooresetmessages: + msg323279
2018-08-08 14:30:16sfx2kcreate