classification
Title: All sys attributes (.argv, ...) should exist in embedded environments
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, ncoghlan, pgacv2, r.david.murray, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2018-01-16 19:46 by pgacv2, last changed 2019-06-02 21:59 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 12463 open Dieter Weber, 2019-05-11 04:53
PR 13553 closed vstinner, 2019-05-24 16:25
Messages (11)
msg310105 - (view) Author: Pedro (pgacv2) Date: 2018-01-16 19:46
Embedded Python interpreters, such as Boost.Python, do not have sys.argv available. (sys itself works fine.) This causes the interpreter to crash with the following exception when you try to access argv:

    AttributeError: 'module' object has no attribute 'argv'

I'm not sure how closely related this is to #15577 or PEP 432. A simple workaround is to manually assign a list with an empty string to argv, as suggested by https://github.com/google/oauth2client/issues/642. However, the documentation for the sys module makes no mention of this special case for argv, and the line at the top that says "It is always available" can easily be interpreted to mean that *all* of sys's attributes will always be available. I suggest adding the following to the documentation for sys.argv:

========

Since `argv` contains the list of **command line** arguments passed to the Python interpreter, `argv` is not available within embedded interpreters, and scripts that run in embedded environments will raise an `AttributeError` when they attempt to access `argv`. As a workaround, assign a list with an empty string manually:
```
import sys

if not hasattr(sys, 'argv'):
    sys.argv  = ['']
```
msg310108 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-01-16 19:56
A better question might be: is there something in the embedding framework that should initialize argv to the empty list?  embedding framework here could have two meanings: either the third party code, or the code that we provide for supporting embedding.
msg310127 - (view) Author: Pedro (pgacv2) Date: 2018-01-17 01:17
My first inclination would be no. An argv value of [''] means "zero command-line arguments" (not even a filename), which implies the following as far as I know:
1. Python was called as an executable rather than a library, because you can't pass command-line arguments to a library
2. No arguments were passed to the executable
3. You are running from inside the REPL

All three are false in an embedded context.

A not-much-better-but-maybe-worth-considering question might also be: should scripts be able to tell whether they are being run from an embedded interpreter? If so, do they have any other way of doing so if an initialization is forced for argv?
msg310128 - (view) Author: Pedro (pgacv2) Date: 2018-01-17 01:18
My comment above uses "code that we provide for supporting embedding," not "third party code," as the definition of embedded.
msg310139 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2018-01-17 05:27
There are actually 3 attributes that may be absent when CPython itself isn't managing the C level entry point: sys.argv, sys.warnoptions, and sys._xoptions.

I'd be more inclined to make the implementation match the docs though, and have these all be empty lists in the not-configured case, rather than absent entirely. (Accessing sys.argv[0] would still be an error in that situation, just an IndexError instead of an AttributeError)
msg310287 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-01-19 20:51
Pedro: David and Nick are proposing initializing sys.argv to [] rather than [''] for embedded interpreters.  This would say 'running embedded'.  The .argv entry would need an additional sentence.

I like Nick's proposal, except that _xoptions should be {}.  Documenting absence should only be a fallback is there is no sensible default.
msg310304 - (view) Author: Pedro (pgacv2) Date: 2018-01-20 00:06
I agree that an empty list, instead of a list with an empty string, makes more sense. Leaving _xoptions as {} would be the same as when it runs non-embedded without any -X arguments, but I guess there's no way of making {} any more empty.
msg310309 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-01-20 00:49
For detecting 'embedded', it is enough that only one documented attribute value be unique to embedded situations.
msg342342 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-13 15:33
In Python 3.8, sys.argv is now always created. It's initialized to sys.argv = [""] if (argc, argv) are not set in _PyCoreConfig.

So PR 12463 looks useless to me, except if someone wants to change the behavior.
msg342343 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-13 15:36
See also the PEP 587.
msg344329 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-06-02 21:59
The issue has been fixed in Python 3.8. See PEP 587 and http://docs.python.org/dev/c-api/init_config.html for the larger scope.

For Python 3.7, the fix is trivial: don't add the following 2 lines in your application:

if not hasattr(sys, 'argv'):
    sys.argv = ['']

Python 3.7 Release Manager, Ned Deily, was opposed to change the behavior in a minor release:
https://github.com/python/cpython/pull/13553#issuecomment-496768319
History
Date User Action Args
2019-06-02 21:59:27vstinnersetstatus: open -> closed
versions: - Python 3.7
messages: + msg344329

resolution: fixed
stage: patch review -> resolved
2019-05-24 16:25:28vstinnersetpull_requests: + pull_request13464
2019-05-13 15:36:03vstinnersetmessages: + msg342343
2019-05-13 15:33:57vstinnersetmessages: + msg342342
2019-05-11 04:53:36Dieter Webersetkeywords: + patch
stage: patch review
pull_requests: + pull_request13153
2018-01-20 00:49:28terry.reedysetmessages: + msg310309
2018-01-20 00:11:57vstinnersetnosy: + vstinner
2018-01-20 00:06:06pgacv2setmessages: + msg310304
2018-01-19 20:51:37terry.reedysettitle: sys.argv documentation should include caveat for embedded environments -> All sys attributes (.argv, ...) should exist in embedded environments
nosy: + terry.reedy

messages: + msg310287

versions: + Python 3.8, - Python 2.7, Python 3.4, Python 3.5, Python 3.6
components: + Library (Lib), - Documentation
2018-01-17 05:27:28ncoghlansetmessages: + msg310139
2018-01-17 01:18:59pgacv2setmessages: + msg310128
2018-01-17 01:17:41pgacv2setmessages: + msg310127
2018-01-16 21:34:14vstinnersetnosy: + ncoghlan
2018-01-16 19:56:50r.david.murraysetnosy: + r.david.murray
messages: + msg310108
2018-01-16 19:46:28pgacv2create