Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the monotonic clock for thread conditions on POSIX platforms #67616

Closed
vstinner opened this issue Feb 9, 2015 · 12 comments
Closed

Use the monotonic clock for thread conditions on POSIX platforms #67616

vstinner opened this issue Feb 9, 2015 · 12 comments
Labels
3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@vstinner
Copy link
Member

vstinner commented Feb 9, 2015

BPO 23428
Nosy @vstinner, @stratakis, @pablogsal, @anikey-m
PRs
  • bpo-32243: Use monotonic clock for thread timeouts; more flexibility for very small switch intervals #4964
  • Superseder
  • bpo-12822: threading.Condition.wait(timeout) should use a monotonic clock: use pthread_condattr_setclock(CLOCK_MONOTONIC)
  • Files
  • cond_timedwait_monotonic.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-02-20.01:01:03.341>
    created_at = <Date 2015-02-09.21:34:27.880>
    labels = ['interpreter-core', '3.8']
    title = 'Use the monotonic clock for thread conditions on POSIX platforms'
    updated_at = <Date 2021-10-01.08:48:32.784>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2021-10-01.08:48:32.784>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-02-20.01:01:03.341>
    closer = 'methane'
    components = ['Interpreter Core']
    creation = <Date 2015-02-09.21:34:27.880>
    creator = 'vstinner'
    dependencies = []
    files = ['38069']
    hgrepos = []
    issue_num = 23428
    keywords = ['patch']
    message_count = 12.0
    messages = ['235637', '235639', '235640', '236156', '273565', '333841', '333844', '333845', '333846', '333851', '336060', '376341']
    nosy_count = 5.0
    nosy_names = ['vstinner', 'neologix', 'cstratak', 'pablogsal', 'anikey']
    pr_nums = ['4964']
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '12822'
    type = None
    url = 'https://bugs.python.org/issue23428'
    versions = ['Python 3.8']

    @vstinner
    Copy link
    Member Author

    vstinner commented Feb 9, 2015

    Python 3.5 now requires a monotonic clock to start and has the C function _PyTime_monotonic().

    Python/condvar.h and Python/thread_pthread.h should use the monotonic clock CLOCK_MONOTONIC, not the system clock CLOCK_REALTIME. See the PEP-418 for the rationale.

    Most platforms support pthread_condattr_setclock(CLOCK_MONOTONIC), except Mac OS X and old versions of Android.

    The glib looks to use pthread_cond_timedwait_relative_np() for Mac OS X:
    https://mail.gnome.org/archives/commits-list/2014-February/msg07782.html

    Note: Android had non-standard pthread_cond_timedwait_monotonic() and pthread_cond_timedwait_monotonic_np() functions. Android is not a official supported platform, and newer Android version now support pthread_condattr_setclock(). I prefer to not support old Android versions (yet).
    https://android-review.googlesource.com/#/c/83881/

    --

    For Windows, SleepConditionVariableSRW() is used on Windows 7 and newer, otherwise WaitForSingleObjectEx() is used. By the way, the check looks to be done during the compilation. I should check which Windows version is used to build Python...

    SleepConditionVariableSRW() and WaitForSingleObjectEx() both take a relative timeout, so they don't use (directly) the system clock. I don't see any required change for Windows.

    According to the PEP-418: "WaitForSingleObject() uses the same timer as GetTickCount() with the same precision."

    Hum, it is not possible to interrupt such wait() with CTRL+c? time.sleep() is implemented with WaitForSingleObjectEx() with _PyOS_SigintEvent(). It doesn't look to be the case here.

    @vstinner vstinner added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Feb 9, 2015
    @vstinner
    Copy link
    Member Author

    vstinner commented Feb 9, 2015

    Python/condvar.h and Python/thread_pthread.h should use the monotonic clock CLOCK_MONOTONIC

    Oh, I forgot that Python/thread_pthread.h only uses pthread_cond_timedwait() if semaphores are emulated with mutexes+conditional variables.

    On most platforms, PyThread_acquire_lock_timed() is implemented with sem_timedwait(). Problem: sem_timedwait() requires an absolute time using the CLOCK_REALTIME clock and the clock is not yet configurable on Linux :-(

    See the feature request in the glibc: "Bug 14717 - Allow choice of clock source for calls to sem_timedwait() and pthread_mutex_timedwait()" opened in 2012:
    https://sourceware.org/bugzilla/show_bug.cgi?id=14717

    Note: QNX provides sem_timedwait_monotonic().

    @vstinner
    Copy link
    Member Author

    vstinner commented Feb 9, 2015

    cond_timedwait_monotonic.patch: Work-in-progress patch. It doesn't change configure.ac yet to check if pthread_condattr_setclock() is supported (with CLOCK_MONOTONIC).

    @vstinner
    Copy link
    Member Author

    Oh, I missed the issue bpo-12822 which looks to fix similar bugs.

    @stratakis
    Copy link
    Mannequin

    stratakis mannequin commented Aug 24, 2016

    Hello,

    Is there any progress on the issue? Should someone take over?

    @vstinner
    Copy link
    Member Author

    I marked bpo-31267 "threading.Timer object is affected by changes to system time: Python locks should use a monotonic clock if available" as a duplicate of this issue.

    @vstinner
    Copy link
    Member Author

    I closed bpo-35747 "Python threading event wait influenced by date change" as a duplicate of the issue.

    @vstinner
    Copy link
    Member Author

    Is there any progress on the issue? Should someone take over?

    It's a limitation of the libc, not directly of Python.

    The problem is that the sem_timedwait() function of the glibc doesn't allow to specify which clock is used:
    https://sourceware.org/bugzilla/show_bug.cgi?id=14717

    Someone has to contribute to the glibc to add an option to sem_init() or sem_timedwait() to allow to use a different clock than CLOCK_REALTIME.

    One workaround is to use Python to use the mutex+cond implementation of pthread locks, since this one is already able to use CLOCK_MONOTONIC:
    https://bugs.python.org/issue31267#msg302257

    @vstinner
    Copy link
    Member Author

    See also bpo-12822: "NewGIL should use CLOCK_MONOTONIC if possible".

    @pablogsal
    Copy link
    Member

    One workaround is to use Python to use the mutex+cond implementation of pthread locks, since this one is already able to use CLOCK_MONOTONIC:

    Does this have any drawbacks?

    @methane methane added the 3.8 only security fixes label Feb 20, 2019
    @methane methane closed this as completed Feb 20, 2019
    @vstinner
    Copy link
    Member Author

    INADA-san fixed bpo-12822 with:

    New changeset 001fee1 by Inada Naoki in branch 'master':
    bpo-12822: use monotonic clock for condvar if possible (GH-11723)
    001fee1

    @vstinner
    Copy link
    Member Author

    vstinner commented Sep 4, 2020

    See bpo-41710 "Timeout is affected by jumps in system time".

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants