Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide sys.executable_argv for host application's command line arguments #74043

Closed
ncoghlan opened this issue Mar 20, 2017 · 14 comments
Closed
Labels
type-feature A feature request or enhancement

Comments

@ncoghlan
Copy link
Contributor

BPO 29857
Nosy @warsaw, @ncoghlan, @vstinner, @jwilk, @stevendaprano
Superseder
  • bpo-23427: Add sys.orig_argv: original command line arguments passed to the Python executable
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-06-08.16:57:23.498>
    created_at = <Date 2017-03-20.06:33:43.103>
    labels = ['type-feature']
    title = "Provide `sys.executable_argv` for host application's command line arguments"
    updated_at = <Date 2020-06-08.16:57:23.496>
    user = 'https://github.com/ncoghlan'

    bugs.python.org fields:

    activity = <Date 2020-06-08.16:57:23.496>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-06-08.16:57:23.498>
    closer = 'vstinner'
    components = []
    creation = <Date 2017-03-20.06:33:43.103>
    creator = 'ncoghlan'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 29857
    keywords = []
    message_count = 14.0
    messages = ['289873', '289885', '289914', '289933', '289934', '289935', '289936', '289937', '289938', '289939', '289940', '289941', '289975', '371016']
    nosy_count = 5.0
    nosy_names = ['barry', 'ncoghlan', 'vstinner', 'jwilk', 'steven.daprano']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '23427'
    type = 'enhancement'
    url = 'https://bugs.python.org/issue29857'
    versions = []

    @ncoghlan
    Copy link
    Contributor Author

    bpo-14208 was ultimately resolved through an import system specific solution, with PEP-451 making the module name passed to python -m available as __main__.__spec__.name.

    However, there are other situations where it may be useful to offer an implementation-dependent attribute in the sys module that provides access to a copy of the host application's raw argv details, rather than the filtered sys.argv details that are left after the host application's command line processing has been completed.

    In the case of CPython, where sys.argv represents the arguments to the Python level main function, sys._raw_argv would be a copy of the argv argument to the C level main() or wmain() function (as appropriate for the platform).

    @ncoghlan ncoghlan added the type-feature A feature request or enhancement label Mar 20, 2017
    @warsaw
    Copy link
    Member

    warsaw commented Mar 20, 2017

    As bytes?

    @ncoghlan
    Copy link
    Contributor Author

    For CPython, I was thinking of having it be "whatever gets passed to Py_Main", and that accepts wchar_t in Py3 [1], so on *Nix systems, the command line has already been decoded with [2] by the time it runs.

    [1] https://docs.python.org/3/c-api/veryhigh.html#c.Py_Main
    [2] https://docs.python.org/3/c-api/sys.html#c.Py_DecodeLocale

    In the case of Windows, the wchar_t array is received straight from the OS as UTF-16-LE.

    @stevendaprano
    Copy link
    Member

    Why is the name flagged as a private implementation detail? I.e. a single leading underscore. I'd be reluctant to rely on this in production code, given how strong the _private convention is.

    Suggest just sys.raw_args instead.

    @vstinner
    Copy link
    Member

    As bytes?

    No, text please. Text is just more convenient in Python, and it's trivial to retrieve original bytes:

    raw_args_bytes = [os.fsencode(arg) for arg in sys._raw_args]

    @warsaw
    Copy link
    Member

    warsaw commented Mar 21, 2017

    On Mar 21, 2017, at 11:47 AM, STINNER Victor wrote:

    No, text please. Text is just more convenient in Python, and it's trivial to
    retrieve original bytes:

    raw_args_bytes = [os.fsencode(arg) for arg in sys._raw_args]

    Well, "raw args" implies minimal or no processing, so bytes would make the
    most sense. It doesn't bother me that it's inconvenient; this won't be an oft
    used API and the conversion to strings should be just as easy.

    @vstinner
    Copy link
    Member

    Well, "raw args" implies minimal or no processing,

    Ok, so call it "original", sys.orig_arv, in that case ;-)

    @vstinner
    Copy link
    Member

    There is already an existing public C API get retrieve original program arguments *as text*:

    /* Make the *original* argc/argv available to other modules.
    This is rare, but it is needed by the secureware extension. */

    void
    Py_GetArgcArgv(int *argc, wchar_t ***argv)
    {
        *argc = orig_argc;
        *argv = orig_argv;
    }

    Are you talking about exposing these arguments at the Python level?

    @ncoghlan
    Copy link
    Contributor Author

    @steven This is an implementation detail in the same sense that sys._getframe() is: it's not something that's actually going to make sense in all contexts. For example, if Py_Main() is never called (for CPython), it would still be None, and other implementations may not define it at all. And even when it's set for CPython, the exact details of what it contains are going to be at least somewhat platform dependent.

    @barry On Windows we define mainw rather than main, so it's the UTF-16-LE encoded text that is the "raw" form. That means the "raw" here refers to "before the Python interpreter CLI processing" - the normalization step to get the command line to wchar_t regardless of platform isn't going to be skipped (since the interpreter runtime itself never even sees the raw bytes in Python 3).

    One option would be to use a longer name like sys._executable_argv, since in the typical case, sys.executable and sys._executable_argv[0] will be the same.

    @victor It wouldn't be exactly the same as Py_GetArgcArgv, as I'd propose making a pristine copy *before* Py_Main() mutates anything - we do some in-place editing of entries while figuring out what "sys.argv[0]" should look like at the Python level.

    @vstinner
    Copy link
    Member

    For example, if Py_Main() is never called (for CPython), it would still be None,

    What is the content of sys.argv in that case? Can't we use the same value for sys._raw_argv?

    @ncoghlan
    Copy link
    Contributor Author

    If the embedding application doesn't call PySys_SetArgv or PySys_SetArgvEx, then there is no argv attribute defined in the sys module (I wasn't actually sure what happened in that case, so I went and checked the code).

    For the reference CLI, the relevant call happens in Py_Main() after all the interpreter level arguments have been processed.

    @vstinner
    Copy link
    Member

    If the embedding application doesn't call PySys_SetArgv or PySys_SetArgvEx, then there is no argv attribute defined in the sys module (I wasn't actually sure what happened in that case, so I went and checked the code).

    Ok, so just don't define sys._raw_argv in that case. But it doesn't seem enough to me to justify to make the attribute private.

    https://docs.python.org/dev/library/sys.html#sys.getallocatedblocks can be seen as an implementation detail.The method name has no underscore prefix, but a simple fallback: return 0 if the feature is is not implemented.

    @ncoghlan
    Copy link
    Contributor Author

    OK, I've changed the proposed attribute name to be sys.executable_argv, with the rationale being that it's "argv as originally seen by sys.executable, rather than by Python's main module"

    As part of documenting this, both it and the argv documentation can make it clear that they may be entirely absent if the host application doesn't set them.

    @ncoghlan ncoghlan changed the title Provide sys._raw_argv for host application's command line arguments Provide sys.executable_argv for host application's command line arguments Mar 22, 2017
    @vstinner
    Copy link
    Member

    vstinner commented Jun 8, 2020

    I mark this issue as a duplicate of bpo-23427 which has patch and is older.

    @vstinner vstinner closed this as completed Jun 8, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants