This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Script’s directory not in sys.path with embeddable Windows distribution
Type: behavior Stage: resolved
Components: Installation Versions:
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Sam Clegg, skoslowski, ssapin, steve.dower
Priority: normal Keywords:

Created on 2018-09-29 07:49 by ssapin, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (8)
msg326671 - (view) Author: Simon Sapin (ssapin) Date: 2018-09-29 07:49
https://docs.python.org/3/library/sys.html#sys.path documents:

> As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter.

On Windows with an embeddable zip file distribution, this does not happen.

Steps to reproduce:

* Create a foo.py file that contains `import bar`
* Create an empty bar.py file
* With your usual Python installed from the "normal" executable installer, check that `python foo.py` runs without output or error
* Download and extract https://www.python.org/ftp/python/3.7.0/python-3.7.0-embed-amd64.zip
* Run `..\python-3.7.0-embed-amd64\python foo.py`

Expected result:

The script runs again without output or error.

Actual result:

Traceback (most recent call last):
  File "foo.py", line 1, in <module>
    import bar
ModuleNotFoundError: No module named 'bar'


This might be an occurrence of https://bugs.python.org/issue33698, since the embeddable distribution has a python37._pth file that contains "python37.zip" and "."

print(sys.path) shows [
  'C:\\Users\\example\\python-3.7.0-embed-amd64\\python37.zip',
  'C:\\Users\\example\\python-3.7.0-embed-amd64'
]

This StackOverflow question describes the same issue: https://stackoverflow.com/q/51403126/1162888
msg326678 - (view) Author: Simon Sapin (ssapin) Date: 2018-09-29 14:00
Removing python37._pth restores the documented behavior, I don’t know if it has adverse effects.
msg336736 - (view) Author: Sebastian Koslowski (skoslowski) * Date: 2019-02-27 08:27
I ran into the same problem.

Seems like adding an empty sys.path entry through the _pth file is not supported at the moment:
https://github.com/python/cpython/blob/53b9e1a1c1d86187ad6fbee492b697ef8be74205/PC/getpathp.c#L589

For now, I have enabled 'import site' and edited that (inside the zip file) to include 'sys.path.insert(0, "")'
msg361378 - (view) Author: Sam Clegg (Sam Clegg) Date: 2020-02-04 23:32
We just ran into the same issue trying to use this zip as part of the emscripten SDK (https://github.com/emscripten-core/emsdk/pull/349).  Seems like a fairly fundamental problem with the windows embeddable zip file.

Is there any downside to simply removing the .pth file?  It seems to have the desired effect.
msg361379 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-02-04 23:49
This is intentional. When embedding the Python runtime, you should limit the import directories to those you want to allow - we do not infer *any* from how it is started, due to the security risks.

Modifying the ._pth file after you extract the distro and before you redistribute your own installer is encouraged, especially if you aren't using an "everything in one directory" layout for your application.

If that statement doesn't sound like it applies to you, then you probably want the regular installer and not the embeddable package. It is *not* meant to be a "portable" distribution.
msg361383 - (view) Author: Sam Clegg (Sam Clegg) Date: 2020-02-05 00:08
In our case we ended up stripping the .pth file and distributing that modified .zip files.

I guess the main problem here is that its really not clear that this version of python doesn't work like the others, or needs to be modified before use.   It can be very confusing to try to debug the resulting error message which for me was:

```
C:\Users\circleci\project\python\3.7.4_64bit\python.exe
Traceback (most recent call last):
  File "C:\Users\circleci\project\upstream\emscripten\\emcc", line 6, in <module>
    from tools import python_selector
ModuleNotFoundError: No module named 'tools'
```

For us we really want a non-interactive installation so a zip file made sense.   We are embeddeding python within another SDK so it sounds like it is applicable.

Maybe this is documented somewhere and I just missed it, but its certainly was a confusing difference from other distributions of python.   

What about adding some kind of startup message such as "This version of python requires local modification before using"?

Also, I'm curious about the kind of security risk there is in including the script directory in sys.path?   If an attacker had access to the script directory couldn't they just modify the script directly?
msg361388 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-02-05 00:26
The install page (https://docs.python.org/3/using/windows.html - linked from the download page) covers non-interactive installs, as well as describing the embeddable distro in detail (https://docs.python.org/3/using/windows.html#windows-embeddable). It sounds like you may be in the right category, but you may also prefer the nuget.org package (https://docs.python.org/3/using/windows.html#windows-nuget), which is intended for arbitrary execution on CI systems rather than constrained execution within another application.

The security concerns are an attacker gaining access to a new machine that they know very little about and searching for a "python.exe" they can use to run their tools. Sure, they can modify their initial script to modify sys.path before trying to import anything, but that's not always possible, so we cut off a number of easy-moderate attacks (and many trivial attacks) by not allowing unanticipated script bundles to be executed.
msg361397 - (view) Author: Sam Clegg (Sam Clegg) Date: 2020-02-05 00:59
Thanks for the info.  If we run into any more difficulties I'll look into nuget.  I should indeed have read the docs more closely.  Apologies.
History
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 79022
2020-02-05 00:59:18Sam Cleggsetmessages: + msg361397
2020-02-05 00:26:05steve.dowersetmessages: + msg361388
2020-02-05 00:08:07Sam Cleggsetmessages: + msg361383
2020-02-04 23:49:08steve.dowersetstatus: open -> closed

nosy: + steve.dower
messages: + msg361379

resolution: not a bug
stage: resolved
2020-02-04 23:35:39Sam Cleggsetcomponents: + Installation
2020-02-04 23:32:21Sam Cleggsetnosy: + Sam Clegg
messages: + msg361378
2019-02-27 08:27:51skoslowskisetnosy: + skoslowski
messages: + msg336736
2018-09-29 14:00:50ssapinsetmessages: + msg326678
2018-09-29 07:49:04ssapincreate