classification
Title: -bb option should override -W options
Type: enhancement Stage: needs patch
Components: Versions: Python 3.7
process
Status: open Resolution: duplicate
Dependencies: Superseder: -W command line options and PYTHONWARNINGS environmental variable should not override -b / -bb command line options
View: 20361
Assigned To: Nosy List: ncoghlan, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017-12-06 10:43 by ncoghlan, last changed 2017-12-06 11:47 by serhiy.storchaka.

Messages (6)
msg307717 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-12-06 10:43
When implementing the "-X dev" mode, Victor was asked to make it behave differently from "-Wd", such that "python -bb -X dev" would still raise errors for bytes comparisons.

I don't think making "-X dev" and "-Wd" inconsistent with each other is the right way to address that request. Instead, I think we should find a way to make "-bb" *always* take precedence over any supplied "-W" options.

One way to do that would be to adopt an approach similar to what I've proposed for "-X dev" in https://bugs.python.org/issue32230: instead of making the warnings module aware of the "-bb" setting, we'd instead adjust the initialisation code to inject "error::BytesWarning" into sys.warnoptions *after* all of the entries from environment variables and the command line.
msg307722 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-12-06 10:52
(From the discussion at https://bugs.python.org/issue32230#msg307721)

Another benefit we'd gain from this approach is that we could easily omit the filter entirely in the case where neither -b nor -bb is set: in those situations, BytesWarning won't be emitted at all, so we don't really need to define a filter for it.
msg307725 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-06 11:04
IMHO the root issue is that there are different options which set warnings filters:

- PYTHONWARNINGS/-W will fill sys.warnoptions
- sys.flags.bytes_warning (-b, -bb) and sys.flags.dev_mode (-X dev, PYTHONDEVMODE=1) which change the default filters

vstinner@apu$ ./python -W default -b -c 'import pprint, sys, warnings; pprint.pprint(warnings.filters); pprint.pprint(sys.warnoptions)'
[('default', None, <class 'Warning'>, None, 0),
 ('default', None, <class 'BytesWarning'>, None, 0),
 ('default', None, <class 'ResourceWarning'>, None, 0)]
['default']

Why default::BytesWarning isn't in sys.warnoptions?

I suggest to experiment to create all filters at once in a single list, rather than having sys.warnoptions on one side and init_filters() (Python/_warnings.c) on another side.

It was on my TODO list for the PEP 432 :-) Currently, when Python is embedded, it's not easy to control filters created by init_filters() which can be an issue. I expect a feature like Py_SetPath() to override *all* warnings filters, not only "add" filters on top on existing filters, for the specific case of embedded Python.
msg307726 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-12-06 11:12
I'm hesitant to put the *true* default filters in sys.warnoptions, as folks use "bool(sys.warnoptions)" as a check for "Were any warning options set via the environment or the command line?", and we don't want to invalidate that use case.

However, I'm definitely a fan of having the warnings module *only* look at sys.warnoptions, and requiring *all* command line options to route their filter settings through that channel.

https://bugs.python.org/issue32230 adjusts "-X dev" to work that way, and I'm suggesting we do the same here for "-b".

That way, we can fully define the relative ordering of PYTHONWARNINGS, "-X dev", "-W", and "-b" inside pymain_add_warnings_options (and only the true default filters will be appended by the warnings module itself).
msg307728 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2017-12-06 11:37
One potential complication here is that embedding applications would inherit the requirement to do both:

    Py_BytesWarningFlag = 2;

and

    PySys_AddWarnOption(L"error::BytesWarning");

to request errors. That's probably OK though, since we're getting into pretty esoteric configuration behaviour at that point.
msg307729 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-06 11:47
This looks like a duplicate of issue20361.
History
Date User Action Args
2017-12-06 11:47:47serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg307729
resolution: duplicate

superseder: -W command line options and PYTHONWARNINGS environmental variable should not override -b / -bb command line options
2017-12-06 11:37:14ncoghlansetmessages: + msg307728
2017-12-06 11:12:24ncoghlansetmessages: + msg307726
2017-12-06 11:04:32vstinnersetmessages: + msg307725
2017-12-06 10:52:25ncoghlansetmessages: + msg307722
2017-12-06 10:43:15ncoghlancreate