classification
Title: Distutils fail if sys.executable is None
Type: behavior Stage: resolved
Components: Distutils Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: dstufft, eric.araujo, iamale, r.david.murray, vstinner
Priority: normal Keywords: patch

Created on 2016-10-28 22:13 by iamale, last changed 2019-04-25 11:16 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12875 merged vstinner, 2019-04-18 14:12
PR 12948 merged miss-islington, 2019-04-25 09:59
PR 12949 merged vstinner, 2019-04-25 10:08
Messages (10)
msg279633 - (view) Author: Alexander P (iamale) Date: 2016-10-28 22:13
For example, Jython 2.7. When we try to execute `jython-standalone -m pip`, distutils.sysconfig tries to parse sys.executable as a path and obviously fails:

    Traceback (most recent call last):
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/runpy.py", line 161, in _run_module_as_main
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/runpy.py", line 72, in _run_code
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__main__.py", line 4, in <module>
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 220, in _main
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 123, in bootstrap
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/ensurepip/__init__.py", line 45, in _run_pip
      File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/__init__.py", line 10, in <module>
      File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/util.py", line 13, in <module>
      File "/tmp/tmp9QnVEm/pip-1.6-py2.py3-none-any.whl/pip/backwardcompat/__init__.py", line 115, in <module>
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/distutils/sysconfig.py", line 28, in <module>
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 367, in realpath
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 373, in _joinrealpath
      File "/home/ale/dev/3toj/jython-standalone-2.7.0.jar/Lib/posixpath.py", line 61, in isabs
    AttributeError: 'NoneType' object has no attribute 'startswith'
msg279653 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-29 04:28
I'm not sure it is reasonable to expect distutils to work if sys.executable is not valid.  What do you think distutils could do instead?
msg279665 - (view) Author: Alexander P (iamale) Date: 2016-10-29 08:46
Well, project_base (which is what sys.executable is used for) is used only during build (i. e. python_build is True), if I understand it correctly. Normally, sys.prefix and sys.exec_prefix are used for all the paths.

Also, project_base can be explicitly set using _PYTHON_PROJECT_BASE environ variable, in which case we still don't need sys.executable (but it still would fail without one right now).
msg279692 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-10-29 18:38
Yes, you are right; I was thinking that distutils and/or pip re-executed python for certain tasks, but upon reflection I don't think they do.
msg279694 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2016-10-29 18:58
We re-execute Python to run setup.py. Even from wheels we do it to compile pyc files. 

Sent from my iPhone

> On Oct 29, 2016, at 2:38 PM, R. David Murray <report@bugs.python.org> wrote:
> 
> 
> R. David Murray added the comment:
> 
> Yes, you are right; I was thinking that distutils and/or pip re-executed python for certain tasks, but upon reflection I don't think they do.
> 
> ----------
> stage:  -> needs patch
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28552>
> _______________________________________
msg340489 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-18 14:13
This issue can be reproduced with:

diff --git a/Lib/site.py b/Lib/site.py
index ad1146332b..c850109c19 100644
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -638,3 +638,5 @@ def _script():
 
 if __name__ == '__main__':
     _script()
+
+sys.executable = None

Attached PR 12875 fix distutils.sysconfig and the distutils build command if sys.executable is None or an empty string. I don't expect that everything works magically, but at least, it's possible to run "./python -m distutils.sysconfig" and use "make" in Python which runs "./python -E ./setup.py build". I'm surprised, but setup.py is able to build C extensions using sys.executable = None :-)

I made a similar fix for sysconfig in bpo-7774: commit 171ba0504aa778d81346ea56fc9000b29d4d3e1d.
msg340829 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-25 09:59
New changeset 0ef8c157e9195df0115c54ba875a5efb92ac22fb by Victor Stinner in branch 'master':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875)
https://github.com/python/cpython/commit/0ef8c157e9195df0115c54ba875a5efb92ac22fb
msg340833 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-25 11:16
New changeset 3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb by Victor Stinner (Miss Islington (bot)) in branch '3.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) (GH-12948)
https://github.com/python/cpython/commit/3076a3e0d1c54a2a6cc54c84521cd0f640d7cffb
msg340835 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-25 11:16
New changeset f4edd39017a211d4544570a1e2ac2110ef8e51b4 by Victor Stinner in branch '2.7':
bpo-28552: Fix distutils.sysconfig for empty sys.executable (GH-12875) (GH-12949)
https://github.com/python/cpython/commit/f4edd39017a211d4544570a1e2ac2110ef8e51b4
msg340836 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-04-25 11:16
I fixed the bug in Python 2.7, 3.7 and master branch (future 3.8).

Thanks Alexander P for the bug report ;-)
History
Date User Action Args
2019-04-25 11:16:46vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg340836

stage: patch review -> resolved
2019-04-25 11:16:07vstinnersetmessages: + msg340835
2019-04-25 11:16:05vstinnersetmessages: + msg340833
2019-04-25 10:08:42vstinnersetpull_requests: + pull_request12874
2019-04-25 09:59:55miss-islingtonsetpull_requests: + pull_request12872
2019-04-25 09:59:54vstinnersetmessages: + msg340829
2019-04-18 14:13:39vstinnersetnosy: + vstinner
messages: + msg340489
2019-04-18 14:12:14vstinnersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request12799
2016-10-29 18:58:08dstufftsetmessages: + msg279694
2016-10-29 18:38:44r.david.murraysetmessages: + msg279692
stage: needs patch
2016-10-29 08:46:15iamalesetmessages: + msg279665
2016-10-29 04:28:02r.david.murraysetnosy: + r.david.murray
messages: + msg279653
2016-10-28 22:13:05iamalecreate