classification
Title: Flag for unusable sys.executable
Type: behavior Stage: resolved
Components: Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: eric.snow, jwilk, ncoghlan, steve.dower
Priority: normal Keywords:

Created on 2018-03-29 14:53 by steve.dower, last changed 2018-09-21 20:44 by steve.dower. This issue is now closed.

Messages (5)
msg314655 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-03-29 14:53
If you host Python in another program, it's likely that sys.executable is not pointing to a normal Python interpreter. This can cause libraries such as multiprocessing to fail when they try to launch the interpreter again. Worse, it may have launched your application many more times before failure :)

I think we should add either a flag to indicate to any such library that sys.executable is not useful for relaunching Python, or a field that points to the actual executable but can safely be left None (or for most horrendous generality, a list of arguments to relaunch, as sometimes a command line option can get you into a normal interpreter). These would be set by embedders only, and Programs/python.c would set the "normal" values.

Thoughts?
msg314656 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-03-29 15:01
Looking at the docs https://docs.python.org/3/library/sys.html#sys.executable maybe we only need to clarify that it's not None only if it provides the normal command line? Then file bugs against all the libraries that don't work when it's not set?
msg314660 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-03-29 15:37
I'm not sure what the right solution is at the design level, but I did recently run into a quirk related to this in the embedding tests: on \*nix systems, PySys_SetProgramName affects sys.executable, while Windows ignores it.

But certainly the intention is that a non-None sys.executable should be usable the way multiprocessing and the test suite use it: as a way to run Python in a subprocess. If an embedding environment wants to support that, it can set it to something suitable, and if it doesn't, it can set it to None.

(I do sometimes wonder if subprocess should have some sys.executable-specific helpers though - rerunning Python with the same settings as the current Python requires quite a bit more than just calling sys.executable with no particular options)
msg314661 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-03-29 15:45
Also mentioning https://docs.python.org/3/library/multiprocessing.html#multiprocessing.set_executable, since it came up on the pickle-protocol-version-5 thread.
msg326036 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-09-21 20:44
Closing in favour of issue34725
History
Date User Action Args
2018-09-21 20:44:42steve.dowersetstatus: open -> closed
resolution: not a bug
messages: + msg326036

stage: resolved
2018-03-30 18:48:27jwilksetnosy: + jwilk
2018-03-29 15:45:40ncoghlansetmessages: + msg314661
2018-03-29 15:37:45ncoghlansetmessages: + msg314660
2018-03-29 15:01:39steve.dowersetmessages: + msg314656
2018-03-29 14:53:31steve.dowercreate