> Please only skip those tests on buildbots.

There is no easy wait to only skip a test on buildbots.

> They work fine otherwise.

The race condition impacts everyone. It's just less likely if your computer is fast enough. The bug is much more likely if you "force" the bad path:

diff --git a/Lib/multiprocessing/semaphore_tracker.py b/Lib/multiprocessing/semaphore_tracker.py
index 3b50a46ddc..7261b43725 100644
--- a/Lib/multiprocessing/semaphore_tracker.py
+++ b/Lib/multiprocessing/semaphore_tracker.py
@@ -107,6 +107,8 @@ getfd = _semaphore_tracker.getfd
 def main(fd):
     '''Run semaphore tracker.'''
     # protect the process from ^C and "killall python" etc
+    import time
+    time.sleep(0.5)
     signal.signal(signal.SIGINT, signal.SIG_IGN)
     signal.signal(signal.SIGTERM, signal.SIG_IGN)
 
diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
index 5c625dd495..6f9f7583e2 100644
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -4193,7 +4193,6 @@ class TestSemaphoreTracker(unittest.TestCase):
         _multiprocessing.sem_unlink(name1)
         p.terminate()
         p.wait()
-        time.sleep(2.0)
         with self.assertRaises(OSError) as ctx:
             _multiprocessing.sem_unlink(name2)
         # docs say it should be ENOENT, but OSX seems to give EINVAL