classification
Title: test.support: add an helper to wait for an event with a timeout
Type: Stage: patch review
Components: Tests Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: pierreglaser, vstinner
Priority: normal Keywords: patch

Created on 2019-05-17 18:21 by vstinner, last changed 2019-05-18 18:37 by pierreglaser.

Pull Requests
URL Status Linked Edit
PR 13407 open pierreglaser, 2019-05-18 18:30
Messages (4)
msg342751 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-17 18:21
The commit cbe72d842646ded2454784679231e3d1e6252e72 is a good example:

            deadline = time.monotonic() + 60
            t = 0.1
            while time.monotonic() < deadline:
                time.sleep(t)
                t = min(t*2, 5)
                try:
                    smm = shared_memory.SharedMemory(name, create=False)
                except FileNotFoundError:
                    break
            else:
                raise AssertionError("A SharedMemory segment was leaked after"
                                     " a process was abruptly terminated.")

It would be nice to convert this code pattern into an helper function in test.support. It's common to have to wait for something in tests.
msg342804 - (view) Author: Pierre Glaser (pierreglaser) * Date: 2019-05-18 17:31
Lib/test/test_asyncio/utils.py defines a similar helper:

def run_until(loop, pred, timeout=30):
    deadline = time.monotonic() + timeout
    while not pred():
        if timeout is not None:
            timeout = deadline - time.monotonic()
            if timeout <= 0:
                raise futures.TimeoutError()
        loop.run_until_complete(tasks.sleep(0.001))

If we trim the ``loop`` usage, we have a rather simple helper can be used to rewrite a decent number of tests, such as:

- _test_multiprocessing.py _TestBarrier._test_reset_f
- _test_multiprocessing.py _TestPoolWorkerLifetime.test_pool_worker_lifetime
- _test_multiprocessing.py TestSyncManagerTypes.test_wait_proc_exit
- fork_wait. ForkWait.test_wait
- test_concurrent_futures.py FailingInitializerMixin.test_initializer
- test_fork1.py ForkTest.waitimpl
- test_logging.py HandlerTests.test_post_fork_child_no_deadlock
- test_os. Win32KillTests._kill
- test_signal.py StressTest.test_stress_delivery_dependent
- test_signal.py StressTest.test_stress_delivery_simulatenous
- test_wait4.py Wait4Test.wait_impl

As well as some top-level commands in:
- test_multiprocessing_main_handling.py some top level instructions
- subprocessdata/sigchlild_ignore.py
- support/__init__.py


I also witnessed some slightly more complex patterns that does not easily fit into the asyncio helper:

# eintr_tester.py FNTREINTLTest._lock
while True:  # synchronize the subprocess
    dt = time.monotonic() - start_time
    if dt > 60.0:
	raise Exception("failed to sync child in %.1f sec" %
dt)
    try:
	lock_func(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
	lock_func(f, fcntl.LOCK_UN)
	time.sleep(0.01)
    except BlockingIOError:
                        break

Which is also (IMO) the case for the lines quoted by Victor.

However, such more complex structures do not seem to appear that often, so sticking to run_until and moving it to test.support.script_helper may be enough.
msg342805 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-05-18 17:45
Would you be interested to propose a PR? Maybe raise an assertion error by
default, but allow to customize it in the API?
msg342808 - (view) Author: Pierre Glaser (pierreglaser) * Date: 2019-05-18 18:37
Just did so.
History
Date User Action Args
2019-05-18 18:37:05pierreglasersetmessages: + msg342808
2019-05-18 18:30:13pierreglasersetkeywords: + patch
stage: patch review
pull_requests: + pull_request13318
2019-05-18 17:45:11vstinnersetmessages: + msg342805
2019-05-18 17:31:52pierreglasersetmessages: + msg342804
2019-05-18 13:08:52pierreglasersetnosy: + pierreglaser
2019-05-17 18:21:48vstinnercreate