While it is nice to have embeddable Python for Windows, currently there seems to be no way of distinguishing it from a "normal" Python, which reacts quite differently to its Environment.

Would like to support Embeddable Python (and ._pth), but really first need a way of identifying whether it is embeddable or not.

Why should a library behave differently if the python using it is embedded in an application?  Can you explain your use case in more detail?

The embedded distribution is meant to be for just that - embedded applications. I'm not quite sure what you mean by support it, or when you would write code that needed to know it was being run from an embedded application.

Can you clarify precisely why you need to be able to get at this information?

I would like to ship Python hermetically with Bazel (embedding it).
For that the Python embedded seems perfect.

A problem arises since currently Bazel is controlling the search path of Python via setting PYTHONPATH (which actually worked with Python 3.5.2-embedded for non prefixed paths).

Now I do not know whether there is any other way of modifying sys.path when calling python.exe.

As a result either

a, have a way to modify sys.path and add support for embeddable
b, have no way to modify sys.path and fail very soon should someone try to use Bazel with a Python embeddable.

Either way the minimum is to have a way to detect which variant it is.

With the embedded distribution, you should probably be calling the Python API rather than running python.exe. And if you do it that way, you can set sys.path via the API before calling user code.

Alternatively, you can set up a site.py within your copy of the embedded distribution to configure the PATH you want.

Or you can just modify the _pth file that's in the embedded distribution. That's pretty much what it's for...

Using the Python API would of course be the preferred way to go.
Sadly that also means that you at minimum have a full compiler and that IMO seems overkill when you only want to build a Python package or a Python test (from the end-user point of view).

The ._pth would be perfect if I could place it alongside the .py script I want to execute.

BTW: Can you please remove PYTHONPATH from the help message and so forth for the embeddable variant. Can be misleading.

I'm still not clear what you're doing here - why does it matter where you have the _pth file?

Could you explain how your application directory is laid out, and what is the main executable for the application? I'm assuming it's a Windows executable, myapp.exe, but the fact that you seem to be implying taht you don't have a C compiler makes me wonder if that's the case.

You could also explain (in the light of the above details) precisely what directories you want to add to sys.path.

Gladly.

So imagine you have the following files:

- foobar
  - pyfoo
    - foo.py
    - src
      - test
        - bar.py

Since Bazel is a build system, you declare (in directory foobar):

py_library(
  name = "foo", # abstract name
  imports = ".",
  srcs = ["pyfoo/foo.py"], # Which source files are in that library
)

py_test(
  name = "bartest",
  srcs = ["pyfoo/src/test/bar.py"], # The test script
  deps = ["foo"], # "Will import foo" -> import of foo's parent directory is added when executing test
)

Bazel does not ship with a specific Python version. So you (the user) start Bazel and provide it with a path to a Python on your system. This means that the Python version MAY be different on every execution of Bazel.

Then you can use Bazel to execute the bartest.
Bazel will create a temporary directory (as a bit of sandboxing) and copy all declared files (+ directories) into it. Bazel will execute the provided Python binary and try set PYTHONPATH so that a import foo does work (notice the imports declaration above).
This is the part where I think a ._pth alongside the script would be beneficial, because for every test invocation the paths will be different. And multiple tests may be executed in parallel (so a "global" ._pth does not cut it). Using a ._pth one could get around setting an environment variable.


Now to simplify deployment I want to put an embeddable Python alongside Bazel and always tell it to use this one. With Embeddable, the only way I have to modify sys.path is to execute Python with -c or write an intermediate script. Both can then modify sys.path and then load bar.py. Obviously this is not as nice as having a direct command-line switch or ._pth mechanism available.

I hope I explained enough. If not - don't hesitate to ask.

OK, well I certainly wouldn't bother supporting users trying to provide the path to an embedded distribution of Python. That's not what the distribution is for, and as the author of Bazel you'd be perfectly OK (IMO) to say you don't support that.

If you want to supply an embedded distribution with Bazel, and deploy it with the application, that's fine - in that case you can modify the _pth file when you copy the distribution to deploy it.

If you want to share the embeddable distribution that is installed with Bazel among multiple user scripts that get deployed, then I'd suggest deploying a wrapper script with the user's code. That wrapper could set sys.path then run the user's script (using the stdlib runpy module, maybe). You mention that option, but say it's "not as nice as having a direct command-line switch" - but while that may be true (I disagree, but it's a matter of opinion) surely the fact that there *is* no such command line switch makes that consideration irrelevant?

Also, I'm not sure how your original question (how you can distinguish an embeddable distribution from a normal one) would help here anyway. FWIW, if you really had to, you could do that by looking to see if there was a _pth file in os.listdir(sys.prefix)). That would be a plausible heuristic, but certainly not something I'd recommend as a robust solution.

I am not the primary contributor to Bazel, so the decision to support embeddable Python is not mine to make.

That said, it seems to me like embeddable Python is a perfect fit if you want to have a hermetic Python installation. And I AM actually the person advocating for support for embeddable Python :).

We should discuss whether we use -c or a wrapper script. The latter might be the better option on the long run for other reasons, too.

My original question stems from the desire to only use e.g. a wrapper script for embeddable for the near future. Reason being that it is not yet as well tested as the PYTHONPATH approach. So would be hesitant to switch from PYTHONPATH to wrapper script for all variants. Also I am not sure yet, the wrapper script is an option for all cases.

Thanks for the input.

You can also use the nuget packages (search on nuget.org for Python). These are intended to be used is scripted standalone setups, such as CI systems. I've been using them as part of most of my build systems recently. They support distutils and pip (not venv in the current version, but that's coming).

You can directly download the nuget.exe tool from https://aka.ms/nugetclidl (e.g. as part of a script).