classification
Title: warnings.simplefilter("always") does not make warnings always show up
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, berker.peksag, exarkun, flox, mark.dickinson, pitrou, python-dev, tseaver
Priority: high Keywords: needs review, patch

Created on 2008-10-22 21:49 by exarkun, last changed 2014-09-18 00:45 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
warn_explicit_replace.patch benjamin.peterson, 2008-10-23 00:49 review
issue4180-py_warnings.diff tseaver, 2010-04-30 02:39
issue4180-test_janitorial.diff tseaver, 2010-04-30 11:05 janitorial patch for FilterTests
issue4180-test_filter_after_default.diff tseaver, 2010-04-30 11:07 new tests for filters after default handling
issue4180-py_warnings2.diff tseaver, 2010-04-30 11:10 Improved python-side handling of filters after default warnings
warnings_issue4180.patch pitrou, 2014-08-21 22:56
Messages (20)
msg75113 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2008-10-22 21:49
If a warning is emitted then a filter with the "always" rule is added
then the is emitted again, it will not be shown the second time:

  exarkun@charm:~$ python
  Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
  [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import warnings
  >>> def f():
  ...     warnings.warn("foo")
  ... 
  >>> f()
  /home/exarkun/.pythonstartup.py:2: UserWarning: foo
  >>> warnings.simplefilter("always")
  >>> f()
  >>> 

The "always" filter is documented as "always print matching warnings"
which is contrary to this behavior.  Also, the string "always" strongly
implies that it will be shown the second time.
msg75117 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-22 22:19
If you didn't raise the warning before using the simple filter, this
would have worked. The undesired behavior is because of
__warningsregistry__. It is set the first time the warning is emitted.
When the second warning comes through, the filter isn't even looked at.
I think the best way to fix this is to invalidate __warningsregistry__
when a filter is used. It would probably be best to store warnings data
in a global then instead of on the module, so it is easy to invalidate.
msg75120 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-10-22 22:36
Obviously the big problem with that change, Benjamin, is you will be
changing the semantics. And that is a tough thing to change when it
involves the warning machinery itself so emitting a warning about a
warning might get messy.
msg75121 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-22 22:39
Oh, do we not consider __warningsregistry__ an implementation detail?
msg75122 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-10-22 22:52
On Wed, Oct 22, 2008 at 3:39 PM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>
> Oh, do we not consider __warningsregistry__ an implementation detail?
>

I guess you could view it that way. I just know I find it extremely
annoying for testing for the exact reasons JP mentions; it's
unexpected if you don't know about it.

If we made __warningsregistry__  about just recording what warnings
have been raised and not include filter information (which I what I
think you are suggesting) then it would still server its purpose,
albeit for probably a small performance penalty. Probably whomever
came up with it over-optimized and just didn't realize that it might
play havoc with testing.
msg75123 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-22 23:11
A simple solution would be to allow warn_explicit to be overridden. Then
a custom warn_explicit function could simply ignore the registry argument.
msg75124 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-10-22 23:16
On Wed, Oct 22, 2008 at 4:11 PM, Benjamin Peterson
<report@bugs.python.org> wrote:
>
> Benjamin Peterson <musiccomposition@gmail.com> added the comment:
>
> A simple solution would be to allow warn_explicit to be overridden. Then
> a custom warn_explicit function could simply ignore the registry argument.

You say "simple" from a comprehension standpoint, I say "pain" in
terms of implementing it in C (although most of the hard stuff I
already have worked out for showwarning).
msg75131 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-10-23 00:49
Don't worry, Brett; you made it really easy. :) Here's a patch.
msg89285 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2009-06-12 15:53
So how about it?
msg89287 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-06-12 16:23
I am still on sabbatical so no code review from me.

But from the standpoint of making warn_explicit be overloadable, I'm on 
the fence. I don't mind it happening, but it will be just one more thing 
we have to support. Since it's Benjamin's code he can make the call to add 
the feature.
msg100525 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-03-06 11:17
+1 for Benjamin's patch, having just been bitten by this exact problem.

I'm trying to do unit testing, checking both that a piece of code produces a DeprecationWarning and that it gives the correct result, with something like:

    with warnings.catch_warnings():
        warnings.filterwarnings("ignore", category=DeprecationWarning)
        self.assertEqual(my_func(my_args), expected_result)

    with warnings.catch_warnings():
        warnings.filterwarnings("error", category=DeprecationWarning)
        self.assertRaises(DeprecationWarning, my_func, *my_args)

The first call still registers the warning, even though it's ignored, so the second assertRaises fails.  Benjamin's patch would seem to provide a way to fix this.

Perhaps I'm missing an obvious better way to do this.

N.B. The above is a too simple version of the real problem: it actually works as intended, for fragile reasons:  the "ignore"d warning is registered on __name__, while the "always"d warning ends up being registered on unittest.case, so there's no conflict.
msg100686 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-03-09 00:45
It sounds like it would be cleaner to apply Brett's idea in msg75122, rather than have the user override (and alternate implementations implement) another obscure hook. We have enough hooks that nobody knows about, IMHO.
msg104610 - (view) Author: Tres Seaver (tseaver) * Date: 2010-04-30 02:39
The attached patch fixes the OP's use case on the Python side by re-ordering the tests, such that "always" prevents the short-circuit from firing::

 $ ./python 
 Python 2.6.5+ (release26-maint, Apr 29 2010, 21:24:12) 
 [GCC 4.3.3] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import warnings as o_warnings
 >>> import sys
 >>> sys.modules['_warnings'] = 0
 >>> del sys.modules['warnings']
 >>> import warnings as py_warnings
 >>> def f():
 ...     py_warnings.warn('foo')
 ... 
 >>> f()
 __main__:2: UserWarning: foo
 >>> f()
 >>> py_warnings.simplefilter('always')
 >>> f()
 __main__:2: UserWarning: foo
 >>> f()
 __main__:2: UserWarning: foo
msg104617 - (view) Author: Tres Seaver (tseaver) * Date: 2010-04-30 11:05
This patch tidies up the FilterWarnings tests to nomalize use of 'self.assertEquals' (matching the rest of the module) and make the 'test_always' assertions meaningful.
msg104618 - (view) Author: Tres Seaver (tseaver) * Date: 2010-04-30 11:07
This patch adds tests for the 'error', 'ignore', and 'always' filters being applied *after* the default warning has been issued, and therefore the registry populated.  It causes failures for the 'error' and 'always' on both the Python and C sides.
msg104619 - (view) Author: Tres Seaver (tseaver) * Date: 2010-04-30 11:10
This patch replaces my earlier 'py_warnings' patch.  It revamps the Python side to check filters before deciding not to emit the warning based on the registry.  The new "<filter>_after_default" tests pass on the Python side with this patch.
msg225618 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-08-21 22:56
Here is a patch implementing an alternate approach, with a version number added in the registry dicts. It also reuses Tres' test cases.

Removing 2.7 because at this point we probably don't want to add non-minimal changes there (outside of the ssl module, that is :-)).
msg226378 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-04 20:28
Brett, do you want to review this?
msg227018 - (view) Author: Roundup Robot (python-dev) Date: 2014-09-18 00:44
New changeset 8adb2c6e0803 by Antoine Pitrou in branch '3.4':
Issue #4180: The warnings registries are now reset when the filters are modified.
https://hg.python.org/cpython/rev/8adb2c6e0803

New changeset 4bc60eb68d3e by Antoine Pitrou in branch 'default':
Issue #4180: The warnings registries are now reset when the filters are modified.
https://hg.python.org/cpython/rev/4bc60eb68d3e
msg227019 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-18 00:45
Ok, this is pushed to 3.x.
History
Date User Action Args
2015-06-24 03:32:35ned.deilylinkissue24490 superseder
2014-09-18 00:45:19pitrousetstatus: open -> closed
resolution: fixed
messages: + msg227019

stage: patch review -> resolved
2014-09-18 00:44:45python-devsetnosy: + python-dev
messages: + msg227018
2014-09-04 20:28:18pitrousetmessages: + msg226378
2014-08-21 22:57:04pitrousetstage: needs patch -> patch review
2014-08-21 22:56:55pitrousetfiles: + warnings_issue4180.patch

messages: + msg225618
versions: - Python 2.7
2014-08-21 16:05:33pitrousetassignee: exarkun ->
stage: patch review -> needs patch
2014-06-05 17:39:17berker.peksagsetnosy: + berker.peksag

versions: + Python 3.4, Python 3.5, - Python 3.2, Python 3.3
2013-01-25 19:14:08brett.cannonsetnosy: - brett.cannon
2011-06-12 21:31:40terry.reedysetversions: + Python 3.3, - Python 3.1
2010-12-14 02:32:28r.david.murraysettype: behavior
versions: + Python 3.1, Python 3.2, - Python 2.6
2010-05-02 17:19:36brett.cannonsetkeywords: + needs review
assignee: exarkun
2010-04-30 11:10:29tseaversetfiles: + issue4180-py_warnings2.diff

messages: + msg104619
2010-04-30 11:07:39tseaversetfiles: + issue4180-test_filter_after_default.diff

messages: + msg104618
2010-04-30 11:05:41tseaversetfiles: + issue4180-test_janitorial.diff

messages: + msg104617
2010-04-30 02:39:13tseaversetfiles: + issue4180-py_warnings.diff

nosy: + tseaver
messages: + msg104610

components: + Library (Lib), - Interpreter Core
type: behavior -> (no value)
2010-03-09 00:45:29pitrousetnosy: + pitrou
messages: + msg100686
2010-03-06 11:20:23floxsetnosy: + flox
stage: patch review

components: + Interpreter Core
versions: + Python 2.7, - Python 2.5, Python 2.4
2010-03-06 11:17:41mark.dickinsonsetnosy: + mark.dickinson
messages: + msg100525
2009-06-12 16:23:26brett.cannonsetmessages: + msg89287
2009-06-12 15:53:44exarkunsetmessages: + msg89285
2008-10-23 00:49:46benjamin.petersonsetfiles: + warn_explicit_replace.patch
keywords: + patch
messages: + msg75131
2008-10-22 23:16:23brett.cannonsetmessages: + msg75124
2008-10-22 23:11:48benjamin.petersonsetmessages: + msg75123
2008-10-22 22:52:14brett.cannonsetmessages: + msg75122
2008-10-22 22:39:07benjamin.petersonsetmessages: + msg75121
2008-10-22 22:36:50brett.cannonsettype: behavior
messages: + msg75120
2008-10-22 22:19:03benjamin.petersonsetpriority: high
nosy: + brett.cannon, benjamin.peterson
messages: + msg75117
2008-10-22 21:49:31exarkuncreate