classification
Title: Executable permissions of Python source files
Type: behavior Stage: resolved
Components: Demos and Tools, Library (Lib) Versions: Python 3.4, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: doko, gregory.p.smith, python-dev, serhiy.storchaka, skrah, tshepang
Priority: normal Keywords: patch

Created on 2013-12-09 11:23 by serhiy.storchaka, last changed 2014-01-23 14:37 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
executable_scripts.patch serhiy.storchaka, 2013-12-09 11:23 review
findnoshebang.py serhiy.storchaka, 2014-01-16 15:44
Messages (14)
msg205677 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-12-09 11:23
Following files have executable permission bit but have no shebang ("#!"):

Lib/test/ssltests.py
Lib/test/test_pathlib.py
Lib/token.py
Tools/scripts/analyze_dxp.py
Tools/scripts/run_tests.py
Tools/scripts/win_add2path.py
Tools/stringbench/stringbench.py

I think executable bit should be cleared on them, because they can't be ran 
without correct shebang.

Following files have shebang but have no executable permission bit:

Doc/includes/email-alternative.py
Doc/includes/email-dir.py
Doc/includes/email-unpack.py
Lib/difflib.py
Lib/http/cookies.py
Lib/idlelib/PyShell.py
Lib/lib2to3/tests/data/different_encoding.py
Lib/lib2to3/tests/data/false_encoding.py
Lib/mailbox.py
Lib/operator.py
Lib/smtplib.py
Lib/tarfile.py
Lib/test/_test_multiprocessing.py
Lib/test/crashers/recursive_call.py
Lib/test/curses_tests.py
Lib/test/multibytecodec_support.py
Lib/test/test___future__.py
Lib/test/test_bz2.py
Lib/test/test_cmd.py
Lib/test/test_codecencodings_cn.py
Lib/test/test_codecencodings_hk.py
Lib/test/test_codecencodings_iso2022.py
Lib/test/test_codecencodings_jp.py
Lib/test/test_codecencodings_kr.py
Lib/test/test_codecencodings_tw.py
Lib/test/test_codecmaps_cn.py
Lib/test/test_codecmaps_hk.py
Lib/test/test_codecmaps_jp.py
Lib/test/test_codecmaps_kr.py
Lib/test/test_codecmaps_tw.py
Lib/test/test_dbm.py
Lib/test/test_dbm_dumb.py
Lib/test/test_eof.py
Lib/test/test_gzip.py
Lib/test/test_keywordonlyarg.py
Lib/test/test_logging.py
Lib/test/test_marshal.py
Lib/test/test_multibytecodec.py
Lib/test/test_popen.py
Lib/test/test_random.py
Lib/test/test_sched.py
Lib/test/test_smtpnet.py
Lib/test/test_socket.py
Lib/test/test_support.py
Lib/test/test_tcl.py
Lib/test/test_urllib2_localnet.py
Lib/test/test_urllib2net.py
Lib/test/test_urllibnet.py
Lib/test/test_with.py
Lib/test/test_xmlrpc_net.py
Lib/timeit.py
Lib/trace.py
Lib/turtledemo/bytedesign.py
Lib/turtledemo/clock.py
Lib/turtledemo/forest.py
Lib/turtledemo/fractalcurves.py
Lib/turtledemo/lindenmayer.py
Lib/turtledemo/minimal_hanoi.py
Lib/turtledemo/paint.py
Lib/turtledemo/peace.py
Lib/turtledemo/penrose.py
Lib/turtledemo/planet_and_moon.py
Lib/turtledemo/tree.py
Lib/turtledemo/two_canvases.py
Lib/turtledemo/yinyang.py
Lib/webbrowser.py
Mac/Tools/bundlebuilder.py
Mac/Tools/plistlib_generate_testdata.py
Modules/_ctypes/libffi/generate-ios-source-and-headers.py
Modules/_ctypes/libffi/generate-osx-source-and-headers.py
Modules/_decimal/tests/bench.py
Modules/_decimal/tests/deccheck.py
Modules/_decimal/tests/randdec.py
Objects/typeslots.py
Tools/clinic/clinic_test.py
Tools/gdb/libpython.py
Tools/i18n/makelocalealias.py
Tools/pybench/Setup.py
Tools/pybench/clockres.py
Tools/pybench/systimes.py
Tools/scripts/checkpip.py
Tools/ssl/make_ssl_data.py
Tools/test2to3/maintest.py
Tools/unicode/comparecodecs.py
Tools/unittestgui/unittestgui.py

I think most of them should got executable bit.

Exceptions are:

Doc/includes/*.py  -- they are just examples, it can be dangerous set 
executable bit on files which can be exposed via http server.

Lib/test/test_*.py -- they don't purposed to ran with with default system 
Python. Shebangs should be removed.

Lib/test/_test_multiprocessing.py, Lib/test/multibytecodec_support.py -- they 
are just auxiliary modules for other tests. Shebangs should be removed.

Lib/http/cookies.py, Lib/mailbox.py, Lib/operator.py, Modules/_decimal/tests/randdec.py -- they don't do anything. Shebangs should be removed.

Tools/test2to3/maintest.py -- this is Python2 script with Python3 shebang.

Lib/difflib.py -- this just runs doctests. Shebang should be removed.

Following files have shebang but have no executable permission bit. Both should be removed.

Lib/test/test_array.py
Lib/test/test_binhex.py
Lib/test/test_errno.py
Lib/test/test_urlparse.py
Lib/test/test_userstring.py

In Tools/unittestgui/unittestgui.py "python" should be changed to "python3" in the shebang.

Any thoughts?
msg208276 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-01-16 12:33
I like this, and I'm doing this in the Debian/Ubuntu packaging anyway.  But I would like to see some check script which maybe can be run before a release, that points out regressions or wrong permissions for newly introduced files.
msg208278 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-16 13:44
New changeset 9062a8695c89 by Stefan Krah in branch '3.3':
Issue #19936: Disable shebang lines in order to prevent using a random
http://hg.python.org/cpython/rev/9062a8695c89
msg208282 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-16 15:37
New changeset 92fd0df03823 by Serhiy Storchaka in branch '3.3':
Issue #19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/92fd0df03823

New changeset 83009d70bc9c by Serhiy Storchaka in branch 'default':
Issue #19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/83009d70bc9c
msg208284 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-16 15:44
Here is a script. It lists text files with executable bits and without shebangs and files with .py, .pyw, or .sh suffix which have shebang but have not with executable bits.

As far as it searches not only Python files, now it found such files:

- ./python-gdb.py
- ./Doc/includes/email-unpack.py
- ./Doc/includes/email-alternative.py
- ./Doc/includes/email-dir.py
- ./Misc/python-config.sh
x ./Mac/PythonLauncher/MyDocument.m
x ./Mac/PythonLauncher/MyDocument.h
x ./Mac/PythonLauncher/FileSettings.m
x ./Mac/PythonLauncher/main.m
x ./Mac/PythonLauncher/FileSettings.h
- ./Tools/test2to3/maintest.py
x ./Tools/scripts/pydocgui.pyw
x ./Lib/idlelib/idle.bat
- ./Modules/_ctypes/libffi/build-ios.sh
x ./Modules/_ctypes/libffi/ltmain.sh
- ./Modules/_ctypes/libffi/msvcc.sh
- ./Modules/_ctypes/libffi/src/arm/gentramp.sh
x ./Modules/_decimal/tests/deccheck.py
x ./Modules/_decimal/tests/runall.bat
x ./Modules/_decimal/tests/bench.py
x ./PC/msvcrtmodule.c
msg208285 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-16 15:49
Somehow Modules/_decimal/tests/bench.py and Modules/_decimal/tests/deccheck.py are now executable, which (I think) they should not be.
msg208287 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-16 16:51
New changeset 085a0ea2015f by Serhiy Storchaka in branch '3.3':
Issue #19936: Remove executable bits from C source files and several forgotten
http://hg.python.org/cpython/rev/085a0ea2015f

New changeset 6baeee39195b by Serhiy Storchaka in branch 'default':
Issue #19936: Remove executable bits from C source files and several forgotten
http://hg.python.org/cpython/rev/6baeee39195b
msg208289 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-16 17:00
New changeset 334116bb2939 by Serhiy Storchaka in branch '2.7':
Issue #19936: Added executable bits or shebang lines to Python scripts which
http://hg.python.org/cpython/rev/334116bb2939
msg208332 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-17 10:44
Left files with mismatched executable bits and shebangs:

Python 3.4:

- ./Doc/includes/email-unpack.py
- ./Doc/includes/email-alternative.py
- ./Doc/includes/email-dir.py
x ./Tools/scripts/pydocgui.pyw
x ./Lib/idlelib/idle.bat
- ./Modules/_ctypes/libffi/build-ios.sh
x ./Modules/_ctypes/libffi/ltmain.sh
- ./Modules/_ctypes/libffi/msvcc.sh
- ./Modules/_ctypes/libffi/src/arm/gentramp.sh
x ./Modules/_decimal/tests/runall.bat

Python 3.3:

- ./Doc/includes/email-unpack.py
- ./Doc/includes/email-alternative.py
- ./Doc/includes/email-dir.py
x ./Lib/idlelib/idle.bat
- ./Modules/_ctypes/libffi/build-ios.sh
x ./Modules/_ctypes/libffi/ltmain.sh
- ./Modules/_ctypes/libffi/msvcc.sh
- ./Modules/_ctypes/libffi/src/arm/gentramp.sh
x ./Modules/_decimal/tests/runall.bat
x ./PC/VS7.1/rt.bat
x ./PC/VC6/rt.bat

Python 2.7:

- ./Doc/includes/email-unpack.py
- ./Doc/includes/email-alternative.py
- ./Doc/includes/email-dir.py
x ./Demo/rpc/test
x ./Lib/idlelib/idle.bat
x ./PC/VS7.1/rt.bat
x ./PC/VC6/rt.bat
msg208335 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-17 13:50
IMO the .bat files should have the executable bit as a reminder that they're
executable on Windows.
msg208694 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-21 19:56
Gregory, is it intentional that Modules/_ctypes/libffi/build-ios.sh, Modules/_ctypes/libffi/libtool-ldflags, Modules/_ctypes/libffi/msvcc.sh and Modules/_ctypes/libffi/src/arm/gentramp.sh have no executable bits? They all have executable bits in mainstream libffi-3.0.13 distribution.
msg208724 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2014-01-22 00:19
For those four libffi files in msg208964, do what you want with them.  I ignored executable bits entirely (didn't set or unset) when importing the code as there is zero need to execute any of them.

IMNSHO, I think executable bits should _not_ be set on most files in the Python source tree.  *Especially* not on things under Lib and Lib/test/.  Afterall, any #! on those is *guaranteed* to not refer to the Python interpreter the source tree represents.
msg208933 - (view) Author: Roundup Robot (python-dev) Date: 2014-01-23 14:24
New changeset 1cfce469642d by Serhiy Storchaka in branch '3.3':
Issue #19936: Restored executable bits for several libffi files.
http://hg.python.org/cpython/rev/1cfce469642d

New changeset 9adce4e25bdb by Serhiy Storchaka in branch 'default':
Issue #19936: Restored executable bits for several libffi files.
http://hg.python.org/cpython/rev/9adce4e25bdb
msg208944 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-01-23 14:37
Now findnoshebang.py shows only example files in the documentation, Windows-specific bat and pyw files, generated files, and the Modules/_ctypes/libffi/ltmain.sh file. I have not touched ltmain.sh file which have no shebang nor executable bits in the libffi distribution, because I'm not sure this is safe.

Thanks all for reviews.
History
Date User Action Args
2014-01-23 14:37:44serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg208944

stage: commit review -> resolved
2014-01-23 14:24:46python-devsetmessages: + msg208933
2014-01-22 00:19:26gregory.p.smithsetmessages: + msg208724
2014-01-21 19:56:08serhiy.storchakasetnosy: + gregory.p.smith
messages: + msg208694
2014-01-17 13:50:40skrahsetmessages: + msg208335
2014-01-17 10:44:14serhiy.storchakasetmessages: + msg208332
2014-01-16 18:45:03serhiy.storchakasetstage: patch review -> commit review
2014-01-16 17:00:11python-devsetmessages: + msg208289
2014-01-16 16:51:36python-devsetmessages: + msg208287
2014-01-16 15:49:46skrahsetnosy: + skrah
messages: + msg208285
2014-01-16 15:44:57serhiy.storchakasetfiles: + findnoshebang.py

messages: + msg208284
2014-01-16 15:37:37python-devsetmessages: + msg208282
2014-01-16 13:44:47python-devsetnosy: + python-dev
messages: + msg208278
2014-01-16 12:33:28dokosetnosy: + doko
messages: + msg208276
2013-12-14 03:24:19tshepangsetnosy: + tshepang
2013-12-09 11:23:21serhiy.storchakacreate