It looks like the warning registry does not get flushed properly:

./python -m test test___all__ test_binhex
0:00:00 load avg: 1.55 Run tests sequentially
0:00:00 load avg: 1.55 [1/2] test___all__
0:00:01 load avg: 1.55 [2/2] test_binhex
test test_binhex crashed -- Traceback (most recent call last):
  File "/Users/remi/src/cpython/Lib/test/libregrtest/runtest.py", line 270, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
  File "/Users/remi/src/cpython/Lib/test/libregrtest/runtest.py", line 221, in _runtest_inner2
    the_module = importlib.import_module(abstest)
  File "/Users/remi/src/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/remi/src/cpython/Lib/test/test_binhex.py", line 10, in <module>
    import binhex
  File "/Users/remi/src/cpython/Lib/contextlib.py", line 124, in __exit__
    next(self.gen)
  File "/Users/remi/src/cpython/Lib/test/support/__init__.py", line 1166, in _filterwarnings
    raise AssertionError("filter (%r, %s) did not catch any warning" %
AssertionError: filter ('', DeprecationWarning) did not catch any warning

test_binhex failed

== Tests result: FAILURE ==

1 test OK.

1 test failed:
    test_binhex

Total duration: 2.0 sec
Tests result: FAILURE



It's not a very issue but does appear when running refleaks on the whole stdlib.

New changeset 9c24e2e4c10705d95258558348417a28007dac66 by Victor Stinner in branch 'master':
bpo-40927: Fix test_binhex when run twice (GH-20764)
https://github.com/python/cpython/commit/9c24e2e4c10705d95258558348417a28007dac66

New changeset af6932575391bca10f4f2145b56e3edbe9765343 by Victor Stinner in branch '3.9':
bpo-40927: Fix test_binhex when run twice (GH-20764) (GH-20789)
https://github.com/python/cpython/commit/af6932575391bca10f4f2145b56e3edbe9765343

Rémi Lapeyre: Thanks for the bug report, it's now fixed!

Oh thanks Irit! It's too easy to forget closing an issue!