classification
Title: -W option and PYTHONWARNINGS env variable does not accept module regexes
Type: Stage: patch review
Components: Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: coldfix, kernc, ncoghlan, nedbat, nikratio, xtreak
Priority: normal Keywords: patch

Created on 2018-09-10 23:36 by coldfix, last changed 2018-12-08 18:35 by kernc.

Pull Requests
URL Status Linked Edit
PR 9358 open coldfix, 2018-09-17 12:43
Messages (6)
msg324959 - (view) Author: Thomas Gläßle (coldfix) * Date: 2018-09-10 23:36
Hi,

This command does not report a warning, while it should:

  python -c 'import warnings; warnings.warn("This should show up")' -Wi -W'default:::.*'

If the regex `.*` is replaced by `__main__` it works as expected.

Same applies for regexes in PYTHONWARNINGS and for the `message` part of the argument. 

The reason can be found in Lib/warnings.py:144 (def _setoption):

  module = re.escape(module)

This point-blank escape makes me think that it was intended that no regexes can be passed to message/module. On the other, the documentation reads as if it should be supported.

Specifically, the -W option is documented in [1]. While this page lists only basic examples, it refers to [2] and [3] for more details. [2] states that message/module are regexes. [3] seems to be written to specifically address the syntax of the PYTHONWARNINGS and the -W option and explicitly lists an example with a regex.

[1]: https://docs.python.org/3/using/cmdline.html#cmdoption-w
[2]: https://docs.python.org/3/library/warnings.html#warning-filter
[3]: https://docs.python.org/3/library/warnings.html#describing-warning-filters

I would welcome if we could remove `re.escape` to make the implementation fit the documentation, or are there any downsides to this?


Best regards, Thomas
msg324960 - (view) Author: Thomas Gläßle (coldfix) * Date: 2018-09-10 23:39
Very sorry, the example command above should read:

  python -Wi -W'default:::.*' -c 'import warnings; warnings.warn("This should show up")'
msg324997 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2018-09-11 08:42
The original escape was done with 2a862c614d6d3ff50dc644150670651fdc9f3a99 which was in 2000. The doc example you are referring to at [1] was added with https://bugs.python.org/issue31975 and there doesn't seem to be a test involving regular expression as I can see from the PR. Adding Nick to the issue since he had committed the doc and might help here.


[1] https://docs.python.org/3/library/warnings.html#describing-warning-filters

Thanks
msg325538 - (view) Author: Thomas Gläßle (coldfix) * Date: 2018-09-17 12:43
Thanks for your response. I have opened a PR at [1] that would remove the re.escape such that the implementation matches the documentation if you decide that this is fine.

[1]: https://github.com/python/cpython/pull/9358

Personally, I would go even further and always add the `(\..*)?$` suffix (instead of only `$`) to make it easier to match all modules in a package which is probably the most important use-case for the case of packages. What do you think?
msg328648 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2018-10-27 13:46
Another option is to make clear in the docs that the command line does not accept regular expressions, but only literal module names, and raise an error if a non-importable name (for example with asterisks in it) is specified.

And while we are here: the docs show "error:::mymodule[.*]" which doesn't even make sense as a regex!
msg328650 - (view) Author: Thomas Gläßle (coldfix) * Date: 2018-10-27 14:10
Hi, thanks for the consideration!

Is there any reason to introduce different behaviour than with filterwarnings here? This increases the number of things to remember - and except for the dot regex syntax doesn't interfere with module names, so there is no big drawback here either.

More importantly, my main use case for filterwarnings would be to select/ignore warnings based on module *or package* name. With the current interpretation as exact module name, you have to list all submodules in advance, which is quite inhibiting.

I have fixed the `[.*]` bogus example in the PR.

Best, Thomas
History
Date User Action Args
2018-12-08 18:35:08kerncsetnosy: + kernc
2018-10-27 14:10:48coldfixsetmessages: + msg328650
2018-10-27 13:46:09nedbatsetnosy: + nedbat
messages: + msg328648
2018-10-07 13:02:32nikratiosetnosy: + nikratio

title: -W option does not accept module regexes -> -W option and PYTHONWARNINGS env variable does not accept module regexes
2018-10-07 12:26:54martin.panterlinkissue34920 superseder
2018-09-17 12:43:34coldfixsetmessages: + msg325538
2018-09-17 12:43:02coldfixsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8781
2018-09-11 08:42:50xtreaksetnosy: + ncoghlan
messages: + msg324997
2018-09-11 08:15:25xtreaksetnosy: + xtreak
2018-09-10 23:39:14coldfixsetmessages: + msg324960
2018-09-10 23:36:34coldfixcreate