If you host Python in another program, it's likely that sys.executable is not pointing to a normal Python interpreter. This can cause libraries such as multiprocessing to fail when they try to launch the interpreter again. Worse, it may have launched your application many more times before failure :)

I think we should add either a flag to indicate to any such library that sys.executable is not useful for relaunching Python, or a field that points to the actual executable but can safely be left None (or for most horrendous generality, a list of arguments to relaunch, as sometimes a command line option can get you into a normal interpreter). These would be set by embedders only, and Programs/python.c would set the "normal" values.

Thoughts?

Looking at the docs https://docs.python.org/3/library/sys.html#sys.executable maybe we only need to clarify that it's not None only if it provides the normal command line? Then file bugs against all the libraries that don't work when it's not set?

I'm not sure what the right solution is at the design level, but I did recently run into a quirk related to this in the embedding tests: on \*nix systems, PySys_SetProgramName affects sys.executable, while Windows ignores it.

But certainly the intention is that a non-None sys.executable should be usable the way multiprocessing and the test suite use it: as a way to run Python in a subprocess. If an embedding environment wants to support that, it can set it to something suitable, and if it doesn't, it can set it to None.

(I do sometimes wonder if subprocess should have some sys.executable-specific helpers though - rerunning Python with the same settings as the current Python requires quite a bit more than just calling sys.executable with no particular options)

Also mentioning https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_executable, since it came up on the pickle-protocol-version-5 thread.

Closing in favour of issue34725