msg113717 - (view) |
Author: Buck Evan (bukzor) * |
Date: 2010-08-12 22:54 |
In our environment, we have a wrapper which enables optimization by default (-OO). Most commandline tools which have a mode-changing flag such as this, also have a flag to do the opposite ( see: ls -t -U, wget -nv -v, ).
I'd like to implement one or both of:
1) Add a -D option which is the opposite of -O. python -OO -D gives an optimization level of 1.
2) Honor PYTHONOPTIMIZE = 0. At the least, the man page needs to describe how these two methods interact.
|
msg114021 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-08-15 23:01 |
How did you conclude that PYTHONOPTIMIZE = 0 is not honored? Can you provide a minimal example or demonstration.
In any case, 2.6.6 is nearly out so bad behavior needs to be demonstrated with 2.7/3.x.
|
msg116996 - (view) |
Author: Buck Evan (bukzor) * |
Date: 2010-09-20 22:59 |
Minimal demo:
$ setenv PYTHONOPTIMIZE 0
$ python3.1 -OO -c "print(__debug__)"
False
I've used this code to get the desired functionality:
if [[ $TESTING == 1 || ${PYTHONOPTIMIZE-2} =~ '^(0*|)$' ]]; then
#someone is requesting no optimization
export -n PYTHONOPTIMIZE
opt=''
elif [[ $PYTHONOPTIMIZE ]]; then
#someone is setting their own optimization
opt=''
else
#optimization by default
opt='-O'
fi
exec $INSTALL_BASE/bin/python2.6 $opt "$@"
|
msg117006 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-09-20 23:39 |
I am following the suggestion in the last sentence of the original post and turning this into a doc issue. I believe it is standard that command-line switches override environmental variables when there is a conflict, but 'Using Python', section 1.2. 'Environment variables' could gain a sentence that says so, or whatever the case is. If I am correct, "These environment variables influence Python’s behavior." could be followed by "They are processed *before* the command-line switches other than -E."
As I read the doc, PYTHONOPTIMIZE = 0, does not seem to mean to the interpreter what you expect it to. "PYTHONOPTIMIZE If this is set to a non-empty string it is equivalent to specifying the -O option. If set to an integer, it is equivalent to specifying -O multiple times." (which is to say, equivalent to -OO.) But the doc could be misleading. You can check the source if you wish.
In general there are lots of switches to alter defaults and turn things on but none to restore defaults and turn things off. The one exception is "-E Ignore all PYTHON* environment variables, e.g. PYTHONPATH and PYTHONHOME, that might be set." The obvious intent is to start fresh and only specify what one wants. I do not think one idiosyncratic wrapper is sufficient reason to add one or more other off switches. There are already about 20 options. If you do not want something turned on, do not turn it on.
|
msg117010 - (view) |
Author: Buck Evan (bukzor) * |
Date: 2010-09-21 00:07 |
If I understand this code, it means that PYTHONOPTIMIZE set to 1 or 2 works as expected, but set to 0, gives a flag value of 1.
static int
add_flag(int flag, const char *envs)
{
int env = atoi(envs);
if (flag < env)
flag = env;
if (flag < 1)
flag = 1;
return flag;
}
Read literally, the man page indicates that any integer value will give a flag value of 2.
I agree my shell script is probably unusual, but I believe setting this environment value to zero and expecting the feature to be off (given no contradicting options) is reasonable.
I petition to remove the second if statement above (environment value of "0" yields no flag).
I'd also love to provide a numeric argument to -O, to dynamically set this value more readily, but that is lower importance.
I can implement these and run the unit tests if required.
|
msg117014 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2010-09-21 01:45 |
I understood the documentation snippet Terry quoted as saying exactly what you say the code does. The doc could be clearer by replacing "multiple times" with "that number of times". Not that more than two has any meaning currently...
Changing the behavior of PYTHONOPTIMIZE as you suggest would be a backward compatibility change that would probably require a deprecation cycle. (And the message would have to not really be a DeprecationWarning, since DeprecationWarnings are now silent by default but this is a user-triggered event, not a developer-triggered event and so should be user visible.)
Unsetting an environment variable to remove its effect is a fairly common idiom. So I don't think it is worth the effort of making the change you suggest, even though I agree it would be more intuitive if PYTHONOPTIMIZE=0 meant what you thought it meant.
|
msg117015 - (view) |
Author: Terry J. Reedy (terry.reedy) * |
Date: 2010-09-21 01:49 |
I do not deny that the behavior described in the doc is a bit strange (keying off of type, ignoring value) is a bit bizarre and contrary to reasonable expectation based on other programs. But to alter behavior before 3.2 or later would require a clearly demonstrated discrepancy between doc and behavior, which I do not see. For startup behavior, some discussion on pydev or python-ideas lists would probably be needed.
If the C code you posted is a quote from the Python source, please give the file path, preferably for current trunk version (the py3k branch). The second 'if' looks a bit strange at first glance, but I will not judge without seeing more.
> I believe setting this environment value to zero and expecting the feature to be off (given no contradicting options) is reasonable.
Optimization is off by default. So there is no need to have a environment value that means to keep it off. Just do not set it. Or unset it if needed.
That said, the code you showed suggests that the option/environment interaction is more complicated than I thought, at least for this pair.
|
msg117019 - (view) |
Author: Buck Evan (bukzor) * |
Date: 2010-09-21 02:11 |
"that number of times" isn't exactly accurate either, since "0" is effectively interpreted as "1".
This change would only adversely affect people who use no -O option, set PYTHONOPTIMIZE to '0', and need optimization.
I feel like that falls into the realm of version differences, but that's your decision.
|
msg117020 - (view) |
Author: Buck Evan (bukzor) * |
Date: 2010-09-21 02:20 |
The file is here:
http://svn.python.org/view/python/trunk/Python/pythonrun.c?view=markup
The second if statement is doing exactly what I find troubling: set the flag even if the incoming value is 0.
I guess this is to handle the empty string case, such as:
setenv PYTHONDEBUG
./myscript.py
|
msg173014 - (view) |
Author: Todd Rovito (Todd.Rovito) * |
Date: 2012-10-16 04:13 |
Applies to Python 3.4 in development and Python 2.7. I recently ran into a similar conflict where I was overriding a environment variable with a command line option and didn't realize it. I first checked the documentation and noticed that it was not clear. Then I found this bug report and made a patch which is attached. The patch iterates what Terry Reedy suggested which I thought made the documentation clearer. This is my first patch to Python I hope it is helpful as Python has served me well over the years.
|
msg173017 - (view) |
Author: Todd Rovito (Todd.Rovito) * |
Date: 2012-10-16 05:17 |
After another review I decided to submit a newer patch that uses a comma so I think the documentation flows better.
|
msg173181 - (view) |
Author: Roundup Robot (python-dev) |
Date: 2012-10-17 14:33 |
New changeset 2f812212599b by Andrew Svetlov in branch '2.7':
Issue #9583: Document startup option/environment interaction.
http://hg.python.org/cpython/rev/2f812212599b
New changeset a8052ad9752b by Andrew Svetlov in branch '3.2':
Issue #9583: Document startup option/environment interaction.
http://hg.python.org/cpython/rev/a8052ad9752b
New changeset f0bb61cbc046 by Andrew Svetlov in branch '3.3':
Merge issue #9583: Document startup option/environment interaction.
http://hg.python.org/cpython/rev/f0bb61cbc046
New changeset def0e0bb06ef by Andrew Svetlov in branch 'default':
Merge issue #9583: Document startup option/environment interaction.
http://hg.python.org/cpython/rev/def0e0bb06ef
|
msg173182 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2012-10-17 14:35 |
Applied patch.
Thanks, Todd.
I think the issue can be closed.
|
msg173193 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2012-10-17 18:41 |
Close the issue.
Feel free to reopen if clarification needed.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:57:05 | admin | set | github: 53792 |
2012-10-17 18:41:42 | asvetlov | set | status: open -> closed resolution: fixed messages:
+ msg173193
stage: needs patch -> resolved |
2012-10-17 14:35:27 | asvetlov | set | nosy:
+ asvetlov
messages:
+ msg173182 versions:
+ Python 3.2, Python 3.3 |
2012-10-17 14:33:41 | python-dev | set | nosy:
+ python-dev messages:
+ msg173181
|
2012-10-16 05:17:23 | Todd.Rovito | set | files:
+ EnvironmentVariables.patch type: enhancement messages:
+ msg173017
|
2012-10-16 04:13:36 | Todd.Rovito | set | files:
+ EnvironmentVariables.patch versions:
+ Python 3.4, - Python 3.1, Python 3.2 nosy:
+ Todd.Rovito
messages:
+ msg173014
keywords:
+ patch |
2010-09-21 02:20:27 | bukzor | set | messages:
+ msg117020 |
2010-09-21 02:11:36 | bukzor | set | messages:
+ msg117019 |
2010-09-21 01:50:00 | terry.reedy | set | messages:
+ msg117015 |
2010-09-21 01:45:03 | r.david.murray | set | nosy:
+ r.david.murray messages:
+ msg117014
|
2010-09-21 00:07:22 | bukzor | set | messages:
+ msg117010 |
2010-09-20 23:40:38 | terry.reedy | set | title: PYTHONOPTIMIZE = 0 is not honored -> Document startup option/environment interaction nosy:
+ docs@python
assignee: docs@python components:
+ Documentation, - Interpreter Core type: behavior -> (no value) |
2010-09-20 23:39:45 | terry.reedy | set | messages:
+ msg117006 stage: needs patch |
2010-09-20 22:59:20 | bukzor | set | messages:
+ msg116996 |
2010-09-20 21:29:03 | eric.araujo | set | versions:
+ Python 3.1 |
2010-08-15 23:01:08 | terry.reedy | set | nosy:
+ terry.reedy
messages:
+ msg114021 versions:
+ Python 2.7, Python 3.2, - Python 2.6 |
2010-08-12 22:54:27 | bukzor | create | |