This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: port shebang of tools from python2 to python3
Type: Stage: patch review
Components: Demos and Tools, Library (Lib) Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, corona10, eric.araujo, ned.deily, petr.viktorin, shihai1991, terry.reedy
Priority: normal Keywords: patch

Created on 2020-08-24 16:28 by shihai1991, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 21948 closed shihai1991, 2020-08-24 16:30
PR 23581 open shihai1991, 2020-12-01 04:12
Messages (8)
msg375855 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-08-24 16:28
After the EOL of python2, the shebang of tools should be ported from python2 to python3.

some files like: https://github.com/python/cpython/blob/master/Objects/typeslots.py#L1
msg379244 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2020-10-21 20:18
This is a minor thing, and not a particularly visible bug, but it could create confusion about how to run scripts so could be fixed.

1) For stdlib modules, I think the intended usage is `python3 -m encodings.rot_13`; I don’t expect anyone to run `python3 /usr/lib/python3.10/encodings/rot_13.py` to do a decoding in the terminal!
So for these easter eggs / hidden gems in the stdlib, I would delete the shebang and remove the executable bit.  There are 18 such modules, ignoring test, lib2to3, turtledemo and a few others named below.

1.1) There may be some exceptions: for Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a symlink to that file, and removing the executable bit would break it.  So this can’t be a mass update but each case should be reviewed.

2) Some of the files in the PR are used during development of CPython: Tools/clinic/clinic.py Python/makeopcodetargets.py Parser/asdl_c.py Objects/typeslots.py Modules/_sha3/cleanup.py Lib/ctypes/macholib/fetch_macholib Mac/BuildScript/build-installer.py
I think some of these work with any Python, but some need the locally built interpreter to give correct results.
It could avoid confusion and mistakes to remove the shebangs and executable bits from the scripts that need a local Python, and make dure their documentation mentions `./python path/to/tool`.

2.1) One recent example, Tools/peg_generator/pegen/__main__.py is documented to work with Python 3.8+, and confusingly is both a __main__ module (to support python3 -m pegen and an executable script with shebang!)

2.2) In Tools/ssl, we have `/usr/bin/env python3` in make_ssl_data.py and `./python` in multissltests.py.  Are these working?!

3) The files in Tools/scripts are documented as «useful for building Python» or «generally useful scripts».  They nearly all executable and nearly all use `#!/usr/bin/env python3`, so fixing the few `python` references there seems ok.  Likewise for Tools/pynche/pynche.

3.1) For Tools/gdb/libpython.py I don’t know how it’s meant to be used; if it’s not distributed but only for debugging a locally built CPython, then removing the shebang seems correct.

(Oh and by the way, Tools/scripts/fixps.py is a tool to replace `/usr/local/bin/python` shebangs with `/usr/bin/env python`…  You can judge if that is useful and/or correct.)


So if things are changed (let’s get a couple opinions here), maybe this needs separate PRs for 1/2/3, and make sure to get reviews to ensure we don’t break IDLE / Mac installers / pegen / etc.
msg379247 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-10-21 20:32
To address some of the concerns:

- The shebang line in Mac/BuildScript/build-installer.py can be safely removed.

- "Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a symlink to that file"
It's not, at least for unix-y builds. In [install-prefix], an "idle*" script is installed of the form:

#![install-prefix]/bin/python3.[n]

from idlelib.pyshell import main
if __name__ == '__main__':
    main()

So there is an absolute link to the correct interpreter and the shebang in pyshell.py is not used.
msg379279 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-10-22 07:06
Thoughts I wrote on the PR that belong here.  (Thanks for the reminder,  Éric.):
*The use of #! in both stdlib and tools seems rather inconsistent.

* Stdlib modules are best run with <python> -m mod so as to run the code with the exact python binary they are meant for. So maybe the marker should be be removed at least from /Lib/*.  But...

-- I was not really aware of idlelib.pyshell.  Running with an explicit  binary seems particularly important for IDLE.  About once a month on SO, some beginner posts about not being able to import a module they downloaded when running IDLE (maybe only sometimes).  Nearly always, they have 2 pythons, such as from Anaconda and python.org.

I may want or need to deprecate using pyshell as entry instead of idle.py/idlew.py/idle.bat as I want to move startup code from pyshell to either idle.py or a separate startup only file.  I am not sure how to get from here to there, partly because I don't really know what 'here' is in practice. 

* With 2.7 put to bed, the line is hardly needed to select between latest 2.x and latest 3.x.

* The response of py.exe to shebang lines needs to be detailed and considered.  I believe it only knows about python.org installs, so it will only start the latest python.org install.  I don't know what it does if 32- and 64- bit versions are both present.  I also don't know what it runs if the 'default' installed version is not the latest installed version.  Does py.exe know what the default is, or does that just affect what 'python' runs?
msg379969 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-10-30 14:27
Sorry for my delay~

>I think some of these work with any Python, but some need the locally built interpreter to give correct results.
>It could avoid confusion and mistakes to remove the shebangs and executable bits from the scripts that need a local Python, and make dure their documentation mentions `./python path/to/tool`.

I update Objects/typeslots.py in #PR21913 recently and using f-format in this script(it means that just local python or python3 can run). So I will accept your suggestion from this file after this PR merged:)

>(Oh and by the way, Tools/scripts/fixps.py is a tool to replace `/usr/local/bin/python` shebangs with `/usr/bin/env python`…  You can judge if that is useful and/or correct.)

Thanks, I will take a look.

> So if things are changed (let’s get a couple opinions here), maybe this needs separate PRs for 1/2/3, and make sure to get reviews to ensure we don’t break IDLE / Mac installers / pegen / etc.

Agree. To corner case, a single sepereate PR is good.
msg379970 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-10-30 14:28
typo error: PR21913->PR21931
msg382238 - (view) Author: Hai Shi (shihai1991) * (Python triager) Date: 2020-12-01 11:23
I create PR23581 to remove the inner Tools' shebang.

But I am not remove Tools/ssl/make_ssl_data.py, Tools/gdb/libpython.py in PR23581.
* To make_ssl_data.py, I guess Christian may give us some guide.
* libpython.py will be installed in kinds of OS, some OS will use or maintain python2.7 for a long times. so I perfer like to keep it.
msg382241 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2020-12-01 11:51
make_ssl_data.py is an internal tool that generates internal header files. It's rarely used. The tool is not bound to any Python version, too. Feel free to change the shebang or leave it as it is. I don't care.
History
Date User Action Args
2022-04-11 14:59:35adminsetgithub: 85792
2021-06-24 15:51:47iritkatriellinkissue39254 superseder
2020-12-01 11:51:42christian.heimessetmessages: + msg382241
2020-12-01 11:24:10shihai1991setnosy: + christian.heimes
2020-12-01 11:23:19shihai1991setmessages: + msg382238
2020-12-01 04:12:41shihai1991setpull_requests: + pull_request22459
2020-10-30 14:28:59shihai1991setmessages: + msg379970
2020-10-30 14:27:19shihai1991setmessages: + msg379969
2020-10-22 07:06:08terry.reedysetnosy: + terry.reedy
messages: + msg379279
2020-10-21 20:32:39ned.deilysetmessages: - msg379245
2020-10-21 20:32:27ned.deilysetmessages: + msg379247
2020-10-21 20:30:43ned.deilysetmessages: + msg379245
2020-10-21 20:18:40eric.araujosetnosy: + petr.viktorin, eric.araujo, ned.deily, corona10
messages: + msg379244
components: + Demos and Tools, Library (Lib)
2020-08-24 16:30:34shihai1991setkeywords: + patch
stage: patch review
pull_requests: + pull_request21058
2020-08-24 16:28:45shihai1991create