classification
Title: pyvenv generates malformed hashbangs for scripts
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: alexreg, ned.deily, ronaldoussoren, vinay.sajip, yan12125
Priority: normal Keywords: patch

Created on 2016-10-15 00:53 by alexreg, last changed 2016-12-27 19:22 by vinay.sajip. This issue is now closed.

Files
File name Uploaded Description Edit
venv-check-path.patch yan12125, 2016-12-16 15:27 review
Messages (4)
msg278676 - (view) Author: Alex Regueiro (alexreg) Date: 2016-10-15 00:53
Quotes around hashbangs are not recognised and are considered invalid syntax, at least on Bash on OS X 10.12. There's really no workaround (that I'm aware of) for paths containing spaces, except maybe symlinking the directory in the path the contains the space. Maybe a warning message about this would be best.

To reproduce this issue, simply run the following from an empty directory that has a space in its path:

```
pyenv venv/
source ./venv/bin/activate
pip
```

The result should be something like:
```
-bash: /Users/me/dir with space/foo/venv/bin/pip: "/Users/me/dir: bad interpreter: No such file or directory
```
msg283410 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2016-12-16 15:27
Seems there are quite a few typos/copy-paste errors in the original steps-to-reproduce. Here are my steps on macOS:

$ ./python-build/python.exe -m venv 'aaa bbb'
$ source ./aaa\ bbb/bin/activate
$ pip
zsh: /Users/yen/tmp/aaa bbb/bin/pip: bad interpreter: "/Users/yen/tmp/aaa: no such file or directory

The result is similar on Linux.

Like Alex said, I don't think there's a portable solution, either, so here's a patch to warn users for possibly broken paths.

macOS kernel has another bug/limitation. In XNU, '#' symbol also breaks shebang:

$ ./python-build/python.exe -m venv 'aaa#bbb'
$ source ./aaa\#bbb/bin/activate
$ pip
zsh: /Users/yen/tmp/aaa#bbb/bin/pip: bad interpreter: /Users/yen/tmp/aaa#bbb/bin/python: no such file or directory

This is even confusing as the kernel thinks the interpreter is /Users/yen/tmp/aaa (everything after the first # is thrown away, see [1]) while zsh/bash thinks it's /Users/yen/tmp/aaa#bbb/bin/python. As a result, I add a warning for #, too

Some related discussions:
1. https://github.com/pypa/pip/issues/923 (pip does not support spaces in directories names)
2. https://github.com/pypa/virtualenv/issues/53 (Whitespace in root path of virtualenv breaks scripts)
3. https://github.com/pypa/virtualenv/issues/973 (bad interpreter error when creating virtualenv's in directories with '#' in name)

Adding venv maintainer and some macOS experts

[1] https://opensource.apple.com/source/xnu/xnu-3789.21.4/bsd/kern/kern_exec.c.auto.html, line 511
msg284107 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2016-12-27 15:19
To reiterate Alex Regueiro's point, I don't think this is a bug, but a shortcoming of the underlying Berkeley exec(2) implementation. See this post:

https://lists.gnu.org/archive/html/bug-bash/2008-05/msg00053.html

We can leave the implementation as is, as on Windows the Python launcher should honour spaces in the interpreter path.

Spaces in shebang lines are not supported where they're not supported by the underlying POSIX implementation - which includes Linux and OS X, AFAICT.
msg284128 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2016-12-27 19:22
As an extra data point, note the behaviour on Windows:

C:\Users\Vinay> \python34\python -m venv "\Temp\aaa bbb"                                                       
                                                                                                               
C:\Users\Vinay> "\Temp\aaa bbb\Scripts\pip" --version                                                          
pip 6.0.8 from C:\Temp\aaa bbb\lib\site-packages (python 3.4)                                                  
                                                                                                               
C:\Users\Vinay> pyzzer -i "\Temp\aaa bbb\Scripts\pip.exe"                                                      
There is a launcher.                                                                                           
Shebang: #!"C:\Temp\aaa bbb\Scripts\python.exe"                                                                
                                                                                                               
                                                                                                               
Archive contents:                                                                                              
  __main__.py                                                                                                  
                                                                                                               
C:\Users\Vinay> "\Temp\aaa bbb\Scripts\python" -m pip install -U pip                                           
You are using pip version 6.0.8, however version 9.0.1 is available.                                           
You should consider upgrading via the 'pip install --upgrade pip' command.                                     
Collecting pip from https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f1
96358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144                                   
  Using cached pip-9.0.1-py2.py3-none-any.whl                                                                  
Installing collected packages: pip                                                                             
  Found existing installation: pip 6.0.8                                                                       
    Uninstalling pip-6.0.8:                                                                                    
      Successfully uninstalled pip-6.0.8                                                                       
                                                                                                               
Successfully installed pip-9.0.1                                                                               
                                                                                                               
C:\Users\Vinay> "\Temp\aaa bbb\Scripts\pip" --version                                                          
pip 9.0.1 from C:\Temp\aaa bbb\lib\site-packages (python 3.4)
History
Date User Action Args
2016-12-27 19:22:24vinay.sajipsetmessages: + msg284128
2016-12-27 15:19:40vinay.sajipsetstatus: open -> closed
resolution: not a bug
messages: + msg284107
2016-12-16 15:27:43yan12125setfiles: + venv-check-path.patch

components: + Library (Lib)
versions: + Python 3.6, Python 3.7
keywords: + patch
nosy: + yan12125, vinay.sajip, ronaldoussoren, ned.deily

messages: + msg283410
2016-10-15 00:53:08alexregcreate