Make installed scripts executable on windows #48265

techtonik · 2008-10-02T12:12:05Z

BPO	4015
Nosy	@mhammond, @amauryfa, @tarekziade, @merwok, @ssbarnea, @asvetlov
Superseder	bpo-12394: packaging: generate scripts from callable (dotted paths)
Files	create.bat.scripts.on.nt.patch executable.scripts.on.nt.patch: version 2 install_scripts.py: version 2 hklm_python_extensions.reg: Registry patch to reister new extensions

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tarekziade'
closed_at = <Date 2011-06-24.12:18:40.746>
created_at = <Date 2008-10-02.12:12:04.817>
labels = ['type-feature']
title = 'Make installed scripts executable on windows'
updated_at = <Date 2011-06-24.12:19:46.278>
user = 'https://bugs.python.org/techtonik'

bugs.python.org fields:

activity = <Date 2011-06-24.12:19:46.278>
actor = 'eric.araujo'
assignee = 'tarek'
closed = True
closed_date = <Date 2011-06-24.12:18:40.746>
closer = 'eric.araujo'
components = ['Distutils2']
creation = <Date 2008-10-02.12:12:04.817>
creator = 'techtonik'
dependencies = []
files = ['11676', '13577', '13578', '17481']
hgrepos = []
issue_num = 4015
keywords = ['patch']
message_count = 24.0
messages = ['74159', '74286', '74311', '74832', '75198', '83624', '83698', '83747', '83750', '83763', '84516', '84988', '84992', '84996', '85001', '85017', '85021', '85086', '85194', '91530', '106671', '106673', '106679', '138932']
nosy_count = 11.0
nosy_names = ['mhammond', 'amaury.forgeotdarc', 'ggenellina', 'techtonik', 'tarek', 'eric.araujo', 'jmfauth', 'ssbarnea', 'phobie', 'bebac', 'asvetlov']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '12394'
type = 'enhancement'
url = 'https://bugs.python.org/issue4015'
versions = ['3rd party']

techtonik · 2008-10-02T12:12:04Z

Distutils contains code to make scripts executable on posix platform.
Here is a patch to for the same feature for nt. It adds .bat file for
every script that doesn't have executable launcher.

terryjreedy · 2008-10-04T00:17:30Z

As a Windows user, I am not sure I would want this. A run command
associated with .py makes all .py files executable. From a command
prompt, which I suspect most Windows users never use, typing 'python' is
not a big deal. Adding .bat files should at least be optional.

techtonik · 2008-10-04T11:16:27Z

Associations still do not show Scripts/ among executable files in Run
dialog.
Association works only for one version of properly installed Python.
It won't work if Python is installed for different user, if extensions
are not registered or if Python was copied from another machine.
Association will call only the latest Python when you may need to
maintain several installations for your applications. 90% of Python
software is still compiled only for Python 2.5 and when .py association
is set for latest 2.6 users still need to have scripts located in 2.5
dist subdir to be executed with version 2.5
Some *nix-developed applications create Python Scripts/ without any
extension at all

Created .bat files guarantee that scripts will be executed with the
version of Python they were installed for. In my opinion this options
should be turned on by default even if made optional to allow users
forget about platform differences.

techtonik · 2008-10-16T11:03:51Z

The same issue in "Roundup Tracker" bugtracker
http://sourceforge.net/tracker2/index.php?func=detail&aid=1163804&group_id=31577&atid=402788

mhammond · 2008-10-24T23:35:35Z

I can see how this might be useful, but I agree it should not happen by
default, at least until it has been out for a while and feedback is
clear that people do want it by default.

I'd also like to find a way to pass all args, not just the first 9 -
that may well end up a source of bugs for people. It might also be
worth considering that setuptools has the ability to create a true
executable from a script by using a stub executable. Wouldn't that be
better still? In the short term, maybe we should keep this
functionality in setuptools etc and let it filter back to distutils as
it proves itself?

bebac · 2009-03-15T09:38:41Z

I think this should be the default. I am a rookie in python, setup.py in
particular, but I cannot see how you can write portable setup scripts
without this.

I agree that the batch script can be improved. Here is how ruby gems do it:

@echo OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
@"ruby.exe" "C:/ruby/bin/fd" %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"ruby.exe" "%~dpn0" %*

amauryfa · 2009-03-17T21:22:05Z

I sometimes use this trick on Windows: name the script with a .bat
extension, and put these lines on top of the file:

@echo off
rem = """
rem run python on this bat file.
rem The -x causes python to skip the first line of the file:
c:\path\to\python -x %~dpnx0 %*
goto :EOF
rem """

bebac · 2009-03-18T13:50:32Z

If you have to name the script with a .bat extension it is not portable
to other platforms or did I misunderstand something?

The point of generating the bat file is to be able to use the same
script on all platforms.

amauryfa · 2009-03-18T14:05:58Z

On posix platform, build_scripts already updates the #! line to refer to
the target interpreter, and changes the file mode.
On Windows, it could change the extension as well. Or does it causes
problems?

bebac · 2009-03-18T17:19:06Z

Ok - I see what you mean. I can't see any problems with it. However
generating a separate bat file has the advantage that you can still
invoke the original script by calling python explicitly.

asvetlov · 2009-03-30T05:58:52Z

optional .bat file generating - probably not bad idea.
But I definitely don't want to see this issue as default.

Maybe just tool for generating bat files for desired packages based on
package metadata for scripts can be solution?

techtonik · 2009-04-01T08:37:18Z

The point is not in generating .bat files. The point is to make scripts
executable with exactly the same version of Python the script was
installed. It works well on POSIX, but doesn't work on windows at all.

There is no other way to fix this on windows than generating separate
.exe or .bat launcher. The patch for .bat is ready.

Embedding python code inside of .bat is not a good idea, because runner
script may be complicated, and additional code in header adds probems
with patch submissions and debugging. Not all editors know about magic
-x option. KISS, you know. =)

jmfauth · 2009-04-01T09:58:04Z

It is true, that on Windows the "mime types", .py, .pyw point to a
specific version of Python.

Having Python 2.4, 2.5, 2.6, 3.0, 3.1 installed on my hd and
applications using these (different) versions, I am *very glad* on that
system, all versions, including \site-packages, are isolated from each
other.

Technically, one can argue a Python developer/user should be able to
write a one line batch file if necessary.

Compared to *x systems, I have always found the Windows way as being
very clean and flexible.

amauryfa · 2009-04-01T11:48:04Z

on Windows the "mime types", .py, .pyw point to a
specific version of Python.

It could also point to a "python launcher", which reads the first line
of the file and starts the corresponding version of the interpreter.
Visual Studio does this for .sln files. It even displays different icons
depending on the file contents.

mhammond · 2009-04-01T12:01:09Z

It could also point to a "python launcher", which reads the first line

What would that first line look like on Windows?
o:\src\python-2.6-svn\PCBuild\python.exe would be appropriate for my
machine, but I wouldn't really be happy with installed scripts embedding
this in their first line - if for no better reason than depending on the
Virtual Machine I am using at the time, that exact same directory will
be seen as being on a completely different device, and potentially under
a different 'root' directory too.

amauryfa · 2009-04-01T15:10:22Z

I agree.
In any case, double-clicking on a .py file should start an "installed"
interpreter, that is one listed in the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\X.Y\InstallPath

Today starting a .py file only open the last installed Python interpreter.
With this proposal, the launcher choose the "best" installed interpreter
for the given script, and falls back to the last installed one.

There may be different definitions of "best". The algorithm could look
like this:

if the first line is
#! c:/some/path/to/python.exe
and the registry contains an installation with
InstallPath="c:/some/path/to"
choose this one.
if the first line matches
#! c:/python/([1-9])([0-9])/python.exe
and if the following registry key exists:
HKEY_LOCAL_MACHINE/SOFTWARE/Python/PythonCore/\1.\2/InstallPath
choose this one.
else, use the last installed interpreter.

asvetlov · 2009-04-01T15:35:51Z

Maybe also let's look on setuptools solution.It can make windows
executable for 'entry point scripts'.

Also there are family scripts for single entry point:

easy_install.exe
easy_install-2.5.exe
easy_install-2.5-script.py
easy_install-script.py

I like it. Exe files executes specific python version. If you are
installed library in python 2.6 - python 2.6 interpreter will be used,
not looking in 'default' interpreter etc.

I use setuptools and for me it gives good 'executive python scripts'.

BTW, double click executes 'default' interpreter, not last installed.

techtonik · 2009-04-01T20:25:44Z

The solution with launcher is complex (if not complicated). It will make
scripts unportable - consider using a removable disk with your Python
and application script. The interpreter was not installed on target
system, but with .bat file application is still able to run.

In case there is a problem with registry (backup/restore/profile
migration operation) when recorded version differs from actual installed
file, you will have a great time trying to debug what your script fails
to work, because the version of Python is wrong. I would name this
behaviour implicit.

techtonik · 2009-04-02T11:13:21Z

I've updated the script to parse unlimited number of parameters on NT,
to return %errorcode% and to fallback to default Python compiler.

It is based on similar workarounds we've made for SCons in
http://scons.tigris.org/source/browse/scons/branches/core/src/script/scons.bat?view=log
If you look at the SCons .bat closely you'll notice the difference that
it includes code to launch main application script directly from
site-packages thus removing the requirement to have .py script in Scripts/

In my patch I use Template that has placeholders for product name,
author and email. However, the actual data is not written and hints how
to get it in place are still welcome.

ssbarnea · 2009-08-13T18:37:13Z

I totally agree that we must create batch files for commands but not by
including python code inside them.

phobie · 2010-05-28T13:52:03Z

On POSIX the interpreter will be read from the first line of a file.
On Windows the interpreter will be read from the Registry HKEY_CLASSES_ROOT\.<file-extension> .

So the correct way to associate a interpreter to a file is to invent a file-extension for every interpreter.
Like /usr/bin/python /usr/bin/python3 and /usr/bin/python3.1 on POSIX, there should be .py .py3 and .py31 on Windows!

I attached a example-registry-patch to register extensions for 2.5, 2.6 and 3.1 .
If you want to use it, you need to adjust the paths!

I propose to change all Python-Windows-installer to install versioned extensions.

If you want a switcher application, it should read the first line of the script and match it against ".*/python(.*)$". So the default POSIX "#!/usr/bin/python3.1" can be kept unchanged. With that rexex the app-path can be read from "HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\<regex-match>\InstallPath\".

BTW.
It would be nice if Python would call itself "Python 3.1" instead of "python" in the "Open with..."-list! The current naming is problematic if you install more than one Python version.

merwok · 2010-05-28T13:58:11Z

Related to bpo-870479 (should we make that one a meta-bug?)

techtonik · 2010-05-28T17:29:30Z

This issue is so old and I do not have time to reread it fully, unfortunately. I believe I wanted to install packages using "easy_install", "pip" or whatever I have and get Scripts/something.bat for my version of Python. This version is often portable, so registry dependency it is not an option - not a good version at least. Scripts/ can also be located in virtualenv.

The similar patch is used for deploying SCons on Windows.

merwok · 2011-06-24T12:19:46Z

We’re going to follow setuptools’ lead and generate platform-appropriate script or binary files from callables.

techtonik mannequin added the stdlib Python modules in the Lib dir label Oct 2, 2008

tarekziade mannequin added the type-feature A feature request or enhancement label Apr 4, 2009

bitdancer changed the title ~~[patch] make installed scripts executable on windows~~ Make installed scripts executable on windows Jun 6, 2010

merwok removed the stdlib Python modules in the Lib dir label Nov 26, 2010

merwok assigned tarekziade Nov 26, 2010

merwok closed this as completed Jun 24, 2011

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make installed scripts executable on windows #48265

Make installed scripts executable on windows #48265

techtonik mannequin commented Oct 2, 2008

techtonik mannequin commented Oct 2, 2008

terryjreedy commented Oct 4, 2008

techtonik mannequin commented Oct 4, 2008

techtonik mannequin commented Oct 16, 2008

mhammond commented Oct 24, 2008

bebac mannequin commented Mar 15, 2009

amauryfa commented Mar 17, 2009

bebac mannequin commented Mar 18, 2009

amauryfa commented Mar 18, 2009

bebac mannequin commented Mar 18, 2009

asvetlov commented Mar 30, 2009

techtonik mannequin commented Apr 1, 2009

jmfauth mannequin commented Apr 1, 2009

amauryfa commented Apr 1, 2009

mhammond commented Apr 1, 2009

amauryfa commented Apr 1, 2009

asvetlov commented Apr 1, 2009

techtonik mannequin commented Apr 1, 2009

techtonik mannequin commented Apr 2, 2009

ssbarnea mannequin commented Aug 13, 2009

phobie mannequin commented May 28, 2010

merwok commented May 28, 2010

techtonik mannequin commented May 28, 2010

merwok commented Jun 24, 2011

Make installed scripts executable on windows #48265

Make installed scripts executable on windows #48265

Comments

techtonik mannequin commented Oct 2, 2008

techtonik mannequin commented Oct 2, 2008

terryjreedy commented Oct 4, 2008

techtonik mannequin commented Oct 4, 2008

techtonik mannequin commented Oct 16, 2008

mhammond commented Oct 24, 2008

bebac mannequin commented Mar 15, 2009

amauryfa commented Mar 17, 2009

bebac mannequin commented Mar 18, 2009

amauryfa commented Mar 18, 2009

bebac mannequin commented Mar 18, 2009

asvetlov commented Mar 30, 2009

techtonik mannequin commented Apr 1, 2009

jmfauth mannequin commented Apr 1, 2009

amauryfa commented Apr 1, 2009

mhammond commented Apr 1, 2009

amauryfa commented Apr 1, 2009

asvetlov commented Apr 1, 2009

techtonik mannequin commented Apr 1, 2009

techtonik mannequin commented Apr 2, 2009

ssbarnea mannequin commented Aug 13, 2009

phobie mannequin commented May 28, 2010

merwok commented May 28, 2010

techtonik mannequin commented May 28, 2010

merwok commented Jun 24, 2011