Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sys.orig_argv: original command line arguments passed to the Python executable #67615

Closed
jgehrcke mannequin opened this issue Feb 9, 2015 · 15 comments
Closed

Add sys.orig_argv: original command line arguments passed to the Python executable #67615

jgehrcke mannequin opened this issue Feb 9, 2015 · 15 comments
Labels
3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement

Comments

@jgehrcke
Copy link
Mannequin

jgehrcke mannequin commented Feb 9, 2015

BPO 23427
Nosy @birkenfeld, @pitrou, @vstinner, @jwilk, @bitdancer, @jgehrcke, @zware
PRs
  • bpo-23427: Add sys.orig_argv attribute #20729
  • Files
  • sys_argv_cmd.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-06-29.22:53:45.902>
    created_at = <Date 2015-02-09.21:09:28.046>
    labels = ['interpreter-core', 'type-feature', '3.10']
    title = 'Add sys.orig_argv: original command line arguments passed to the Python executable'
    updated_at = <Date 2020-06-29.22:53:45.901>
    user = 'https://github.com/jgehrcke'

    bugs.python.org fields:

    activity = <Date 2020-06-29.22:53:45.901>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-06-29.22:53:45.902>
    closer = 'vstinner'
    components = ['Interpreter Core']
    creation = <Date 2015-02-09.21:09:28.046>
    creator = 'jgehrcke'
    dependencies = []
    files = ['38065']
    hgrepos = []
    issue_num = 23427
    keywords = ['patch']
    message_count = 15.0
    messages = ['235633', '235645', '235677', '235680', '235681', '235705', '371017', '371022', '371025', '371027', '371028', '371031', '372157', '372639', '372640']
    nosy_count = 9.0
    nosy_names = ['georg.brandl', 'pitrou', 'vstinner', 'jwilk', 'r.david.murray', 'jgehrcke', 'docs@python', 'zach.ware', 'krivushinme']
    pr_nums = ['20729']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue23427'
    versions = ['Python 3.10']

    @jgehrcke
    Copy link
    Mannequin Author

    jgehrcke mannequin commented Feb 9, 2015

    When Python is invoked with the -c command switch, the command string does not get exposed in sys.argv:

        $ python -c "import sys; print(sys.argv)"
        ['-c']
    
        $ python -c "import sys; print(sys.argv)" arg1
        ['-c', 'arg1']

    The command string does not get exposed anywhere, AFAIK, so it is inaccessible from within Python programs. There might be application scenarios in which it is useful to access the command string, such as for debugging purposes. One scenario is when a Python session should be able to "re-spawn" itself in a subprocess (I came across this question on StackOverflow: http://stackoverflow.com/q/28412903/145400)

    I propose to make the command string accessible. If you agree that it might make sense, the question is *how/where* to expose it.

    One possible way is to retain it in sys.argv, as in this example:

        $ python -c "import sys; print(sys.argv)" "arg1"
        ['-c', 'import sys; print(sys.argv)', 'arg1']

    The current sys.argv docs say

    If the command was executed using the -c command line option to
    the interpreter, argv[0] is set to the string '-c'.

    This sentence could then be adjusted to

    "[...], argv[0] is set to the string '-c', and argv[1] contains the command."

    This method breaks existing applications that are started with the -c method and that consume command line arguments in a sys.argv[1:] fashion. The tests in Lib/test/test_cmd_line.py all pass, however.

    A second method would be to change sys.argv[0] from '-c' to '-c command'. This would break existing applications that check for sys.argv[0] == 'c'.

    A third method would be to leave sys.argv as it is, and expose the command with a new attribute in the sys module.

    I have attached a patch for variant 1 (passes all tests in Lib/test/test_cmd_line.py), to demonstrate which code is affected: the translation from the "real" argv to sys' argv is triggered in Modules/main.c. The patch does not change behavior of '-m' (it's funny, however, that the current version of main.c at first replaces the module string with '-m', whereas the runpy module later on replaces '-m' with the path to the module file anyway.).

    As a side node, I figure that the sys.argv documentation should be adjusted to properly reflect the -m behavior, which is:

        $ ./python -m testmodule foo
        testmodule sys.argv: ['/data/local/pythondev/pythontip/cpython/testmodule.py', 'foo']

    Let me hear your comments, and I am willing to work on code and doc patches, thanks!

    @jgehrcke jgehrcke mannequin assigned docspython Feb 9, 2015
    @jgehrcke jgehrcke mannequin added docs Documentation in the Doc dir interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement labels Feb 9, 2015
    @vstinner
    Copy link
    Member

    vstinner commented Feb 9, 2015

    sys.argv must not be changed. It would break too many Python applications.

    *If* we decide to expose the command line parameter in Python, we can
    add a new variable like sys.command for example. "command" name in
    used in the C code of Python, and also comes from "c" of "-c".

    @krivushinme
    Copy link
    Mannequin

    krivushinme mannequin commented Feb 10, 2015

    Hello, I have find some workaround to get actual argv, but it broken:

    python -c 'import ctypes; argv = ctypes.POINTER(ctypes.c_char_p)(); argc = ctypes.c_int(); ctypes.pythonapi.Py_GetArgcArgv(ctypes.byref(argc), ctypes.byref(argv)); print([argv[i] for i in xrange(0, argc.value)])'

    And this will output:
    ['python', '-c', '-c']

    May be we just need to fix this behaviour, due this is error, as far as i can see. But may broke something.

    @jgehrcke
    Copy link
    Mannequin Author

    jgehrcke mannequin commented Feb 10, 2015

    Victor,

    I support the idea of sys.command. However, it would be unpopulated most of the time (e.g. set to None by default). Now, is that something we should push forward or not? I would work on a patch, but we should have an agreement first, I guess.

    Mihail,

    the original argv becomes modified in the very early bootstrap phase, and the command gets lost within that process: it gets *overwritten* with "-c", which is exactly why you are observing two "-c". This happens here:

    https://hg.python.org/cpython/file/default/Modules/main.c#l684

    So, no, without a code change in main.c there will be no way to retain the command for later usage.

    @krivushinme
    Copy link
    Mannequin

    krivushinme mannequin commented Feb 10, 2015

    Jan-Philip, yes, I see that Main.c needs modification, but we can fix orig_argv with not just assignment but with full copy. So then we can get unmodified argv.

    @zware
    Copy link
    Member

    zware commented Feb 10, 2015

    Rather than add a variable to sys that will be empty 99% of the time, I think I'd rather stick a '__command__' constant in the __main__ module namespace when running with '-c' (think of '__file__'). You could then get at it elsewhere with 'from __main__ import __command__' (probably wrapped in a try/except ImportError, since it will usually not exist).

    This should probably be discussed on python-ideas.

    (Removing all versions but 3.5, as this is a feature request.)

    @zware zware removed the docs Documentation in the Doc dir label Feb 10, 2015
    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    I marked bpo-29857 as a duplicate of this issue.

    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    See also bpo-14208 "No way to recover original argv with python -m". For the specific case of python -m, the original argument has been available as __main__.__spec__.name since Python 3.4.

    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    Many names have been proposed:

    I chose "sys.orig_argv" attribute name with the documentation:

    The list of the original command line arguments passed
    to the Python executable.

    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    I marked bpo-15577 "Real argc and argv in embedded interpreter" as duplicate of this issue: my PR 20729 allows embedders to set PyConfig.orig_argv which becomes sys.orig_argv.

    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    Example of sys.orig_argv usage to re-execute the Python process with different options:
    ---

    import sys
    import os
    if not sys.flags.utf8_mode:
        # Force UTF-8 mode
        argv = sys.orig_argv.copy()
        argv[1:1] = ["-X", "utf8"]
        print(f"Re-execute to force UTF-8 mode! argv={argv}")
        os.execv(argv[0], argv)
    
    print(f"Everybody loves UTF-8! utf8_mode={sys.flags.utf8_mode}")

    Example coming from discussions on the PEP-597 :-)

    Output:
    ---

    $ ./python force_utf8_mode.py 
    Re-execute to force UTF-8 mode! argv=['./python', '-X', 'utf8', 'force_utf8_mode.py']
    Everybody loves UTF-8! utf8_mode=1

    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    My implementation (PR 20729) is based on bpo-40910 change which added a private PyConfig._orig_argv member to fix Py_GetArgcArgv().

    @vstinner vstinner changed the title Python should expose command when invoked with -c Add sys.orig_argv: original command line arguments passed to the Python executable Jun 8, 2020
    @vstinner vstinner added the 3.10 only security fixes label Jun 16, 2020
    @vstinner
    Copy link
    Member

    The setproctitle project uses Py_GetArgcArgv() and would benefit of PyConfig.orig_argv, see:

    @vstinner
    Copy link
    Member

    New changeset dd8a93e by Victor Stinner in branch 'master':
    bpo-23427: Add sys.orig_argv attribute (GH-20729)
    dd8a93e

    @vstinner
    Copy link
    Member

    I added sys.orig_argv to the master branch (future Python 3.10).

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants