classification
Title: test_asyncio.test_events.GetEventLoopTestsMixin.test_get_event_loop_new_process mixin breaks in the Unix environment without working /dev/shm
Type: Stage: resolved
Components: asyncio Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, mcepl, miss-islington, ned.deily, vstinner, yselivanov
Priority: normal Keywords: patch

Created on 2019-10-05 13:47 by mcepl, last changed 2020-06-21 21:17 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 16593 closed mcepl, 2019-10-05 13:48
PR 19073 closed mcepl, 2020-03-19 15:02
PR 20944 merged vstinner, 2020-06-17 16:10
PR 20962 merged vstinner, 2020-06-18 13:01
PR 20966 merged vstinner, 2020-06-18 15:25
PR 20984 merged vstinner, 2020-06-19 14:59
PR 20987 merged miss-islington, 2020-06-19 16:01
PR 20988 merged miss-islington, 2020-06-19 16:01
Messages (12)
msg354010 - (view) Author: Matej Cepl (mcepl) * Date: 2019-10-05 13:47
======================================================================
ERROR: test_get_event_loop_new_process (test.test_asyncio.test_events.TestCGetEventLoop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2647, in test_get_event_loop_new_process
    self.loop.run_until_complete(main()),
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/asyncio/base_events.py", line 608, in run_until_complete
    return future.result()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2640, in main
    pool = concurrent.futures.ProcessPoolExecutor()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 555, in __init__
    self._call_queue = _SafeQueue(
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 165, in __init__
    super().__init__(max_size, ctx=ctx)
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/queues.py", line 42, in __init__
    self._rlock = ctx.Lock()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 38] Function not implemented

======================================================================
ERROR: test_get_event_loop_new_process (test.test_asyncio.test_events.TestPyGetEventLoop)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2647, in test_get_event_loop_new_process
    self.loop.run_until_complete(main()),
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/asyncio/base_events.py", line 608, in run_until_complete
    return future.result()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2640, in main
    pool = concurrent.futures.ProcessPoolExecutor()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 555, in __init__
    self._call_queue = _SafeQueue(
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 165, in __init__
    super().__init__(max_size, ctx=ctx)
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/queues.py", line 42, in __init__
    self._rlock = ctx.Lock()
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
OSError: [Errno 38] Function not implemented

----------------------------------------------------------------------

The problem is that in the OpenBuildService (the build system of SUSE distributions) /dev/shm is limited to:

abuild@milic:~/rpmbuild/BUILD/Python-3.8.0rc1> ls -ld /dev/shm
drwxr-xr-x 1 root root 0 Oct  5 13:23 /dev/shm
abuild@milic:~/rpmbuild/BUILD/Python-3.8.0rc1>

Provided PR will skip test if /dev/shm doesn’t exist or it doesn’t sufficient permissions.
msg371809 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-18 12:53
New changeset ddbeb2f3e02a510c5784ffd74c5e09e8c70b5881 by Victor Stinner in branch 'master':
bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944)
https://github.com/python/cpython/commit/ddbeb2f3e02a510c5784ffd74c5e09e8c70b5881
msg371819 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-18 15:20
New changeset b1e736113484c99acb57e4acb417b91a9e58e7ff by Victor Stinner in branch '3.9':
bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944) (GH-20962)
https://github.com/python/cpython/commit/b1e736113484c99acb57e4acb417b91a9e58e7ff
msg371826 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-18 16:56
New changeset e8056180a13b6755e4e3e5505b7bf03f79da29fb by Victor Stinner in branch '3.8':
bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944) (GH-20962) (GH-20966)
https://github.com/python/cpython/commit/e8056180a13b6755e4e3e5505b7bf03f79da29fb
msg371828 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-18 16:58
Hum, test_resource_tracker_reused() failed twice on macOS, but it's unclear to me if it's a regression caused by my change or not.

x86-64 macOS 3.9:
https://buildbot.python.org/all/#builders/696/builds/119

FAIL: test_resource_tracker_reused (test.test_multiprocessing_forkserver.TestResourceTracker)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/_test_multiprocessing.py", line 5219, in test_resource_tracker_reused
    self.assertTrue(is_resource_tracker_reused)
AssertionError: False is not true
msg371833 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-06-18 19:29
> Hum, test_resource_tracker_reused() failed twice on macOS, but it's unclear to me if it's a regression caused by my change or not.

It does appear to be caused by your change. I did a quick check with the 3.8 branch. With that PR, test_multiprocessing_forkserver fails solidly. Backing up to the previous commit, test_multiprocessing_forkserver passes.
msg371836 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-18 21:01
test_multiprocessing_forkserver.test_resource_tracker_reused() failed on 3.8, 3.9 and master buildbots. It looks like a real reproducible bug. I'm not sure how my change triggered this bug.
msg371886 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-19 14:55
From what I understood, the problem is that creating a SemLock has an impact on the resource tracker.

The failing unit tests checks something about the resource tracker.

The test runs a function in a child process. The function is defined in _test_multiprocessing.py. _test_multiprocessing.py calls skip_if_broken_multiprocessing_synchronize() which creates a SemLock(). At the end, the child process modifies the resource tracker because it creates a SemLock().

There are different options to fix the issue:

* Modify skip_if_broken_multiprocessing_synchronize() only only attempt to create a SemLock on Linux.
* Modify skip_if_broken_multiprocessing_synchronize() to only check if the current user is allowed to write into /dev/shm/ directory: that would allow touching the resource tracker.
* Modify the test to avoid improting _test_multiprocessing in the child process.

I don't know why the test only fails on macOS.

Note: I ignored macOS failure when I merged my PR because the macOS job of GitHub Action is failing randomly for at least 3 months and nobody attempted to fix it. So I learnt to ignore this job.
msg371891 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-19 16:01
New changeset 3358da4054b9b0b045eb47dc74dee3d58bfbb1d5 by Victor Stinner in branch 'master':
bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)
https://github.com/python/cpython/commit/3358da4054b9b0b045eb47dc74dee3d58bfbb1d5
msg371892 - (view) Author: miss-islington (miss-islington) Date: 2020-06-19 16:19
New changeset ec9bc2da421c456e416d991fd1fe79ac33344d9d by Miss Islington (bot) in branch '3.8':
bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)
https://github.com/python/cpython/commit/ec9bc2da421c456e416d991fd1fe79ac33344d9d
msg371893 - (view) Author: miss-islington (miss-islington) Date: 2020-06-19 16:20
New changeset 1529322ca6b7b029af65aceeb04e6d47013eeeb4 by Miss Islington (bot) in branch '3.9':
bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)
https://github.com/python/cpython/commit/1529322ca6b7b029af65aceeb04e6d47013eeeb4
msg372021 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-06-21 21:17
Ok, macOS buildbots are back to green, I close the issue.
History
Date User Action Args
2020-06-21 21:17:29vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg372021

stage: patch review -> resolved
2020-06-19 16:20:19miss-islingtonsetmessages: + msg371893
2020-06-19 16:19:45miss-islingtonsetmessages: + msg371892
2020-06-19 16:01:47miss-islingtonsetpull_requests: + pull_request20162
2020-06-19 16:01:40miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request20161
2020-06-19 16:01:25vstinnersetmessages: + msg371891
2020-06-19 14:59:43vstinnersetpull_requests: + pull_request20158
2020-06-19 14:55:19vstinnersetmessages: + msg371886
2020-06-18 21:01:35vstinnersetmessages: + msg371836
versions: + Python 3.10
2020-06-18 19:29:42ned.deilysetnosy: + ned.deily
messages: + msg371833
2020-06-18 16:58:55vstinnersetmessages: + msg371828
2020-06-18 16:56:52vstinnersetmessages: + msg371826
2020-06-18 15:25:02vstinnersetpull_requests: + pull_request20144
2020-06-18 15:20:08vstinnersetmessages: + msg371819
2020-06-18 13:01:02vstinnersetpull_requests: + pull_request20141
2020-06-18 12:53:27vstinnersetmessages: + msg371809
2020-06-17 16:10:44vstinnersetnosy: + vstinner
pull_requests: + pull_request20125
2020-03-19 15:02:18mceplsetpull_requests: + pull_request18428
2019-10-05 13:48:15mceplsetkeywords: + patch
stage: patch review
pull_requests: + pull_request16182
2019-10-05 13:47:18mceplcreate