classification
Title: Many multiprocessing tests are silently skipped since 3.9
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: davin, lukasz.langa, miss-islington, pitrou, serhiy.storchaka, sobolevn, vstinner
Priority: high Keywords: 3.9regression, patch

Created on 2021-08-29 11:01 by serhiy.storchaka, last changed 2021-09-20 07:49 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28060 merged sobolevn, 2021-08-30 11:53
PR 28168 merged miss-islington, 2021-09-04 20:42
PR 28169 merged serhiy.storchaka, 2021-09-04 20:51
PR 28182 sobolevn, 2021-09-07 14:26
Messages (13)
msg400521 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-29 11:01
Here is a list of multiprocessing tests which are run in 3.8 but are not found in 3.9+:

OtherTest.test_answer_challenge_auth_failure
OtherTest.test_deliver_challenge_auth_failure
TestInitializers.test_manager_initializer
TestInitializers.test_pool_initializer
TestSyncManagerTypes.test_array
TestSyncManagerTypes.test_barrier
TestSyncManagerTypes.test_bounded_semaphore
TestSyncManagerTypes.test_condition
TestSyncManagerTypes.test_dict
TestSyncManagerTypes.test_event
TestSyncManagerTypes.test_joinable_queue
TestSyncManagerTypes.test_list
TestSyncManagerTypes.test_lock
TestSyncManagerTypes.test_namespace
TestSyncManagerTypes.test_pool
TestSyncManagerTypes.test_queue
TestSyncManagerTypes.test_rlock
TestSyncManagerTypes.test_semaphore
TestSyncManagerTypes.test_value
WithManagerTestBarrier.test_abort
WithManagerTestBarrier.test_abort_and_reset
WithManagerTestBarrier.test_action
WithManagerTestBarrier.test_barrier
WithManagerTestBarrier.test_barrier_10
WithManagerTestBarrier.test_default_timeout
WithManagerTestBarrier.test_reset
WithManagerTestBarrier.test_single_thread
WithManagerTestBarrier.test_thousand
WithManagerTestBarrier.test_timeout
WithManagerTestBarrier.test_wait_return
WithManagerTestCondition.test_notify
WithManagerTestCondition.test_notify_all
WithManagerTestCondition.test_notify_n
WithManagerTestCondition.test_timeout
WithManagerTestCondition.test_wait_result
WithManagerTestCondition.test_waitfor
WithManagerTestCondition.test_waitfor_timeout
WithManagerTestContainers.test_dict
WithManagerTestContainers.test_dict_iter
WithManagerTestContainers.test_dict_proxy_nested
WithManagerTestContainers.test_list
WithManagerTestContainers.test_list_iter
WithManagerTestContainers.test_list_proxy_in_list
WithManagerTestContainers.test_namespace
WithManagerTestEvent.test_event
WithManagerTestLock.test_lock
WithManagerTestLock.test_lock_context
WithManagerTestLock.test_rlock
WithManagerTestManagerRestart.test_rapid_restart
WithManagerTestMyManager.test_mymanager
WithManagerTestMyManager.test_mymanager_context
WithManagerTestMyManager.test_mymanager_context_prestarted
WithManagerTestPool.test_apply
WithManagerTestPool.test_async
WithManagerTestPool.test_async_timeout
WithManagerTestPool.test_context
WithManagerTestPool.test_empty_iterable
WithManagerTestPool.test_enter
WithManagerTestPool.test_imap
WithManagerTestPool.test_imap_handle_iterable_exception
WithManagerTestPool.test_imap_unordered
WithManagerTestPool.test_imap_unordered_handle_iterable_exception
WithManagerTestPool.test_make_pool
WithManagerTestPool.test_map
WithManagerTestPool.test_map_async
WithManagerTestPool.test_map_async_callbacks
WithManagerTestPool.test_map_chunksize
WithManagerTestPool.test_map_handle_iterable_exception
WithManagerTestPool.test_map_no_failfast
WithManagerTestPool.test_map_unplicklable
WithManagerTestPool.test_release_task_refs
WithManagerTestPool.test_resource_warning
WithManagerTestPool.test_starmap
WithManagerTestPool.test_starmap_async
WithManagerTestPool.test_terminate
WithManagerTestPool.test_traceback
WithManagerTestPool.test_wrapped_exception
WithManagerTestQueue.test_closed_queue_put_get_exceptions
WithManagerTestQueue.test_fork
WithManagerTestQueue.test_get
WithManagerTestQueue.test_no_import_lock_contention
WithManagerTestQueue.test_put
WithManagerTestQueue.test_qsize
WithManagerTestQueue.test_queue_feeder_donot_stop_onexc
WithManagerTestQueue.test_queue_feeder_on_queue_feeder_error
WithManagerTestQueue.test_task_done
WithManagerTestQueue.test_timeout
WithManagerTestRemoteManager.test_remote
WithManagerTestSemaphore.test_bounded_semaphore
WithManagerTestSemaphore.test_semaphore
WithManagerTestSemaphore.test_timeout
WithProcessesTestManagerRestart.test_rapid_restart
WithProcessesTestPicklingConnections.test_access
WithProcessesTestPicklingConnections.test_pickling
WithProcessesTestSharedMemory.test_shared_memory_ShareableList_basics
WithProcessesTestSharedMemory.test_shared_memory_ShareableList_pickling
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_basics
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_reuses_resource_tracker
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryServer_ignores_sigint
WithProcessesTestSharedMemory.test_shared_memory_across_processes
WithProcessesTestSharedMemory.test_shared_memory_basics
WithProcessesTestSharedMemory.test_shared_memory_cleaned_after_process_termination
WithThreadsTestManagerRestart.test_rapid_restart
msg400587 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-08-30 11:48
Looks like that this happens because tests classes are decorated with `@hashlib_helper.requires_hashdigest('md5')`: https://github.com/python/cpython/blob/2a8127cafe1d196f858a3ecabf5f1df3eebf9a12/Lib/test/_test_multiprocessing.py#L4482-L4485

Tests above run fine without this decorator.
It was added in https://bugs.python.org/issue17258

The problem is that `` was treating a decorated entity as a function, not a class. https://github.com/python/cpython/blob/e6497fe698f6e87344501a68ffdea106eafcb257/Lib/test/support/hashlib_helper.py#L25 But, it was decorating classes as well.
So, it needs to changed to respect classes.

I will make a PR shortly.
msg400588 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-08-30 12:15
Wow. I hope this didn't hide any regression :-(
msg400590 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-08-30 12:36
Looks like it did, two newly-unignored tests are failing:
https://github.com/python/cpython/pull/28060#issuecomment-908295411

пн, 30 авг. 2021 г. в 15:16, Antoine Pitrou <report@bugs.python.org>:

>
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> Wow. I hope this didn't hide any regression :-(
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue45042>
> _______________________________________
>
msg400594 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-30 13:13
Thank you Nikita.

The difference between the original and the proposed in PR 28060 code is that in the original code the conditional was tested at the testing time, while in the proposed code it is tested at the loading time. I do not know what effect  it causes on tests. If it does not matter, the code can be simpler:

   return unittest.skipIf(should_be_skipped,
                          f"hash digest '{digestname}' is not available.")

But if it matters, there are two options:

1. In requires_hashdigest() raise exception if func is a class. Remove the decorator from classes and add it to test and setup methods or just to the setUpClass() classmethod.

2. In requires_hashdigest() check if func is a class, and if it is true, patch its test and setup methods (or just add a decorated setUpClass() classmethod).
msg400597 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-08-30 13:20
Here is the implementation of the second option:

def requires_hashdigest(digestname, openssl=None, usedforsecurity=True):
    def decorator(func):
        if isinstance(func, type):
            setUpClass = func.__dict__.get('setUpClass')
            if setUpClass is None:
                def setUpClass(cls):
                    super(func, cls).setUpClass()
                setUpClass.__qualname__ = func.__qualname__ + '.setUpClass'
                setUpClass.__module__ = func.__module__
            else:
                setUpClass = setUpClass.__func__
            setUpClass = classmethod(decorator(setUpClass))
            func.setUpClass = setUpClass
            return func
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            try:
                if openssl and _hashlib is not None:
                    _hashlib.new(digestname, usedforsecurity=usedforsecurity)
                else:
                    hashlib.new(digestname, usedforsecurity=usedforsecurity)
            except ValueError:
                raise unittest.SkipTest(
                    f"hash digest '{digestname}' is not available."
                )
            return func(*args, **kwargs)
        return wrapper
    return decorator
msg400605 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-08-30 13:47
Serhiy, yes, you are right. I guess, it is safe to assume that load-time/test-time might make a difference, especially with `_hashlib`.

I've commited your suggestion, thanks a lot for your help!
msg400612 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2021-08-30 14:33
FYI, there seem to be two Windows-specific regressions since the tests were unintentionally disabled, namely test_shared_memory_basics and test_checksum_fodder.

Following Serhiy's advice, I elect to have those tests skipped for now on Windows and fix them through separate issues. We will be releasing 3.9.7 according to schedule today.
msg400650 - (view) Author: Nikita Sobolev (sobolevn) * Date: 2021-08-30 18:23
Two new issues created:
1. https://bugs.python.org/issue45053
2. https://bugs.python.org/issue45052
msg401060 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-04 20:42
New changeset dd7b816ac87e468e2fa65ce83c2a03fe1da8503e by Nikita Sobolev in branch 'main':
bpo-45042: Now test classes decorated with `requires_hashdigest` are not skipped (GH-28060)
https://github.com/python/cpython/commit/dd7b816ac87e468e2fa65ce83c2a03fe1da8503e
msg401061 - (view) Author: miss-islington (miss-islington) Date: 2021-09-04 21:04
New changeset e5976dd2e6e966183da59df99978ebcb4b3a32df by Miss Islington (bot) in branch '3.10':
bpo-45042: Now test classes decorated with `requires_hashdigest` are not skipped (GH-28060)
https://github.com/python/cpython/commit/e5976dd2e6e966183da59df99978ebcb4b3a32df
msg401072 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-09-05 06:41
New changeset ab58269ab3b784bef33a613cd0a68914065a9134 by Serhiy Storchaka in branch '3.9':
[3.9] bpo-45042: Now test classes decorated with `requires_hashdigest` are not skipped (GH-28060) (GH-28169)
https://github.com/python/cpython/commit/ab58269ab3b784bef33a613cd0a68914065a9134
msg402204 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-09-20 07:49
See also bpo-45128 (fixed): "test_multiprocessing_fork fails if run sequentially after test_genericalias and test_logging".
History
Date User Action Args
2021-09-20 07:49:54vstinnersetnosy: + vstinner
messages: + msg402204
2021-09-17 20:14:25serhiy.storchakalinkissue42389 superseder
2021-09-07 14:26:35sobolevnsetpull_requests: + pull_request26631
2021-09-05 06:42:38serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2021-09-05 06:41:40serhiy.storchakasetmessages: + msg401072
2021-09-04 21:04:51miss-islingtonsetmessages: + msg401061
2021-09-04 20:51:25serhiy.storchakasetpull_requests: + pull_request26598
2021-09-04 20:42:46miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request26597
2021-09-04 20:42:44serhiy.storchakasetmessages: + msg401060
2021-08-31 21:53:36vstinnersetnosy: - vstinner
2021-08-30 18:23:56sobolevnsetmessages: + msg400650
2021-08-30 14:33:39lukasz.langasetnosy: + lukasz.langa
messages: + msg400612
2021-08-30 13:47:05sobolevnsetmessages: + msg400605
2021-08-30 13:20:29serhiy.storchakasetmessages: + msg400597
2021-08-30 13:13:37serhiy.storchakasetmessages: + msg400594
2021-08-30 12:36:22sobolevnsetmessages: + msg400590
2021-08-30 12:15:38pitrousetmessages: + msg400588
2021-08-30 11:53:25sobolevnsetkeywords: + patch
stage: patch review
pull_requests: + pull_request26504
2021-08-30 11:48:02sobolevnsetnosy: + sobolevn
messages: + msg400587
2021-08-29 11:01:45serhiy.storchakasetpriority: normal -> high
2021-08-29 11:01:30serhiy.storchakacreate