Sometimes we have the following problem:

• there are way too many deprecations raised from a given library to silence them one by one;
• the deprecations are different between maintenance branches and we don't want to make tests between the branches hopelessly conflicting.

In particular, there is such a case right now with asyncio in 3.9 vs. later branches. 3.8 deprecated the loop= argument in a bunch of functions but due to poor warning placement, most of them were silent. This is being fixed in BPO-44815 which would go out to users in 3.9.7 but that created 220 new warnings when running test_asyncion in regression tests. Fixing them one by one would be both tedious, and would make the 3.9 branch forever conflicting with newer branches in many asyncio test files. In 3.11 there's a new round of deprecations raised in test_asyncio, making the branches different. Moreover, those warnings are typically silenced by assertWarns context managers which should only be used when actually testing the warnings, not to silence irrelevant warnings.

So, what the PR does is it introduces:

• support.ignore_deprecations_from("path.to.module", like=".*msg regex.*"), and
• support.clear_ignored_deprecations()

The former adds a new filter to warnings, the message regex is mandatory. The latter removes only the filters that were added by the former, leaving all other filters alone.

Example usage is in test_support, and later, should this be merged, will be in asyncio tests on the 3.9 branch.

For example, all that's needed to silence the 220 new warnings in all asyncio tests, is adding this in Lib/test/test_asyncio/init.py:

def setUpModule():
    support.ignore_deprecations_from("asyncio.base_events", like=r".*loop argument.*")
    support.ignore_deprecations_from("asyncio.unix_events", like=r".*loop argument.*")
    support.ignore_deprecations_from("asyncio.futures", like=r".*loop argument.*")
    support.ignore_deprecations_from("asyncio.runners", like=r".*loop argument.*")
    support.ignore_deprecations_from("asyncio.subprocess", like=r".*loop argument.*")
    support.ignore_deprecations_from("asyncio.tasks", like=r".*loop argument.*")
    support.ignore_deprecations_from("test.test_asyncio.test_queues", like=r".*loop argument.*")
    support.ignore_deprecations_from("test.test_asyncio.test_tasks", like=r".*loop argument.*")

def tearDownModule():
    support.clear_ignored_deprecations()

Since the __init__.py file in question isn't a module that runs tests itself but only a package gathering many sub-modules, we need some boilerplate like:

def load_tests(loader, _, pattern):
    pkg_dir = os.path.dirname(__file__)
    suite = AsyncioTestSuite()
    return support.load_package_tests(pkg_dir, loader, suite, pattern)


class AsyncioTestSuite(unittest.TestSuite):
    """A custom test suite that also runs setup/teardown for the whole package.

    Normally unittest only runs setUpModule() and tearDownModule() within each
    test module part of the test suite. Copying those functions to each file
    would be tedious, let's run this once and for all.
    """
    def run(self, result, debug=False):
        try:
            setUpModule()
            super().run(result, debug=debug)
        finally:
            tearDownModule()

With that, all of asyncio tests silence unnecessary deprecation warnings. Additionally, testing for warnings with warnings_helper.check_warnings() or assertWarns still works just fine as those facilities temporarily disable filtering.

There are problems with clearing all ignored deprecation filters.

We can want to ignore warnings in narrower or wider scope: per-method, per-class, per-module. We should use the warnings.catch_warnings() context manager for restoring filters in the correct scope. For example, use setUp/tearDown, setUpClass/tearDownClass, setUpModule/tearDownModule and save the context manager as an instance/class attribute or module global.

   def setUp(self):
       self.w = warnings.catch_warnings()
       self.w.__enter__()
       warnings.filterwarnings(...)

    def tearDown():
        self.w.__exit__(None, None, None)

It is hard to use any helper here because the code should be added in multiple places.

Or use setUp/addCleanup, setUpClass/addClassCleanup, setUpModule/addModuleCleanup if you want to keep all code in one place:

   def setUp(self):
       w = warnings.catch_warnings()
       w.__enter__()
       self.addCleanup(w.__exit__, None, None, None)
       warnings.filterwarnings(...)

The helper can call catch_warnings(), __enter__(), set filters and return a caller which calls __exit__(None, None, None):

   def setUp(self):
       self.addCleanup(some_helper(...))

For class and method scopes it would be convenient to use class and method decorators correspondingly.

    @ignore_some_warnings(...)
    def test_foo(self):
        ...

@ignore_some_warnings(...)
class BarTests(unittest.TestCase):
    ...

Serhiy, if you take a look at the code, clear_ignored_deprecations() only clears the filters that were added by ignore_deprecations_from(). It doesn't touch other filters. Sure, we can abuse the context manager instead but this looks pretty busy:

   def setUp(self):
       self.w = warnings.catch_warnings()
       self.w.__enter__()
       warnings.filterwarnings(...)

    def tearDown():
        self.w.__exit__(None, None, None)

Making the API symmetrical is a good idea though, I'll look into that tomorrow.

clear_ignored_deprecations() does not know whether filters were added in setUp(), setUpClass(), setUpModule() or just in a particular method. We should use teadDown*() or add*Cleanup() to clean up at appropriate level.

New changeset a0a6d39 by Łukasz Langa in branch 'main':
bpo-44852: Support ignoring specific DeprecationWarnings wholesale in regrtest (GH-27634)
a0a6d39

New changeset c7c4cbc by Łukasz Langa in branch '3.9':
[3.9] bpo-44852: Support ignoring specific DeprecationWarnings wholesale in regrtest (GH-27634) (GH-27785)
c7c4cbc

New changeset bc98f98 by Łukasz Langa in branch '3.10':
[3.10] bpo-44852: Support ignoring specific DeprecationWarnings wholesale in regrtest (GH-27634) (GH-27784)
bc98f98

New changeset 8cf07d3 by Łukasz Langa in branch 'main':
bpo-44852: Support filtering over warnings without a set message (GH-27793)
8cf07d3

New changeset d1c0e44 by Miss Islington (bot) in branch '3.10':
bpo-44852: Support filtering over warnings without a set message (GH-27793)
d1c0e44

New changeset ebe7e6d by Miss Islington (bot) in branch '3.9':
bpo-44852: Support filtering over warnings without a set message (GH-27793) (GH-27810)
ebe7e6d