Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime #63207

Closed
pitrou opened this issue Sep 12, 2013 · 11 comments
Closed

precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime #63207

pitrou opened this issue Sep 12, 2013 · 11 comments
Labels
3.11 only security fixes OS-windows stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@pitrou
Copy link
Member

pitrou commented Sep 12, 2013

BPO 19007
Nosy @malemburg, @tim-one, @abalkin, @pitrou, @vstinner, @tjguk, @4kir4, @zware, @eryksun, @zooba
Files
  • time_precise.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2013-09-12.19:44:34.318>
    labels = ['type-feature', 'library', 'OS-windows', '3.11']
    title = 'precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime'
    updated_at = <Date 2021-06-13.22:48:09.141>
    user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2021-06-13.22:48:09.141>
    actor = 'eryksun'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)', 'Windows']
    creation = <Date 2013-09-12.19:44:34.318>
    creator = 'pitrou'
    dependencies = []
    files = ['40088']
    hgrepos = []
    issue_num = 19007
    keywords = ['patch']
    message_count = 9.0
    messages = ['197538', '197539', '197541', '224358', '247024', '247778', '247779', '247808', '395767']
    nosy_count = 10.0
    nosy_names = ['lemburg', 'tim.peters', 'belopolsky', 'pitrou', 'vstinner', 'tim.golden', 'akira', 'zach.ware', 'eryksun', 'steve.dower']
    pr_nums = []
    priority = 'low'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue19007'
    versions = ['Python 3.11']

    Linked PRs

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 12, 2013

    For whoever is interested: Windows 8 apparently has a new API named GetSystemTimePreciseAsFileTime which returns the system API with a much better resolution than GetSystemTimeAsFileTime does ("The GetSystemTimePreciseAsFileTime function retrieves the current system date and time with the highest possible level of precision (<1us)").
    http://msdn.microsoft.com/en-us/library/windows/desktop/hh706895%28v=vs.85%29.aspx

    (spawned from this topic on the tulip mailing-list: https://groups.google.com/forum/#!topic/python-tulip/vX9vOZB1FOI )

    @pitrou pitrou added stdlib Python modules in the Lib dir OS-windows type-feature A feature request or enhancement labels Sep 12, 2013
    @vstinner
    Copy link
    Member

    time.time() is sometimes used in performance critical code. Is GetSystemTimePreciseAsFileTime() as fast as GetSystemTimeAsFileTime()?

    Linux has the opposite: CLOCK_REALTIME_COARSE. This clock is less accurate but may be faster.
    http://lwn.net/Articles/342018/

    @pitrou
    Copy link
    Member Author

    pitrou commented Sep 12, 2013

    "GetSystemTimePreciseAsFileTime() has very little overhead; the new function is even a little faster than GetSystemTimeAsFileTime(), a call takes a few ten nanoseconds."

    from http://www.windowstimestamp.com/description#C_2

    @vstinner
    Copy link
    Member

    My patch for the issue bpo-22043 replaces the _PyTime_timeval structure (10^-6 second resolution, 1 us) with a _PyTime_timespec structure (10^-9 second resolution, 1 ns). It may help for this issue.

    @vstinner vstinner changed the title precise time.time() under Windows 8 precise time.time() under Windows 8: use GetSystemTimePreciseAsFileTime Jul 30, 2014
    @vstinner
    Copy link
    Member

    Python 3.5 has a new C function _PyTime_GetSystemClock() which uses the new _PyTime_t type. It now has a resolution of 1 nanosecond.

    Sorry, I don't have Windows 8 at home, so I cannot work on this issue.

    I guess that we should check at runtime if GetSystemTimePreciseAsFileTime() is available, as we did for GetTickCount64() in the past:

    hKernel32 = GetModuleHandleW(L"KERNEL32");
    *(FARPROC*)&Py_GetTickCount64 = GetProcAddress(hKernel32,
                                                   "GetTickCount64");

    Is there any Windows developer interested to write a short patch to implement it?

    @vstinner
    Copy link
    Member

    Good news! I got a new fresh Windows 8.1 VM with Visual Studio 2015. I'm now able to work on this issue.

    I wrote a patch: time_precise.patch.

    Resolution computed in Python by https://hg.python.org/peps/file/tip/pep-0418/clock_resolution.py:

    GetSystemTimePreciseAsFileTime(): 715 ns
    GetSystemTimeAsFileTime(): 14 ms

    Obviously, the resolution is better...

    GetSystemTimePreciseAsFileTime() uses internally the QueryPerformanceCounter() so I chose to use QueryPerformanceFrequency() to fill time.get_clock_info('time').resolution, same code than time.get_clock_info('perf_counter').resolution

    @vstinner
    Copy link
    Member

    Note: GetSystemTimePreciseAsFileTime() is restricted to desktop applications.

    The windowstimestamp.com has a warning on this function:

    http://www.windowstimestamp.com/description

    """
    2.1.4.2. Desktop Applications: GetSystemTimePreciseAsFileTime()

    (...)
    The function shall also be used with care when a system time adjustment is active. Current Windows versions treat the performance counter frequency as a constant. The high resolution of GetSystemTimePreciseAsFileTime() is derived from the performance counter value at the time of the call and the performance counter frequency. However, the performance counter frequency should be corrected during system time adjustments to adapt to the modified progress in time. Current Windows versions don't do this. The obtained microsecond part may be severely affected when system time adjustments are active. Seconds may consist of more or less than 1.000.000 microseconds. Microsoft may or not fix this in one of the next updates/versions.
    (...)
    As of May, 2015 the inaccuracy of GetSystemTimePreciseAsFileTime() during system time adjustments persists for the preview versions of Windows 10.
    """

    Is it ok to switch to GetSystemTimePreciseAsFileTime() for Python time.time()?

    @zooba
    Copy link
    Member

    zooba commented Aug 1, 2015

    What are the expected benefits from changing? Just a higher resolution? I'm not sure that's worth anything if it's inaccurate.

    @eryksun
    Copy link
    Contributor

    eryksun commented Jun 13, 2021

    What are the expected benefits from changing? Just a higher
    resolution? I'm not sure that's worth anything if it's inaccurate.

    GetSystemTimePreciseAsFileTime() returns an accurate timestamp, which is the current system time plus the difference between the current value of the performance counter and its value when the system time was last updated. This can't be replicated simply with separate calls to GetSystemTimeAsFileTime() and QueryPerformanceCounter(). It relies on the following undocumented fields in the KUSER_SHARED_DATA record: BaselineSystemTimeQpc, QpcSystemTimeIncrement, and QpcSystemTimeIncrementShift.

    The system uses a similar procedure in QueryInterruptTimePrecise() to extend the precision of QueryInterruptTime(). Ditto for QueryUnbiasedInterruptTime() vs QueryUnbiasedInterruptTimePrecise().

    "GetSystemTimePreciseAsFileTime() has very little overhead; the new
    function is even a little faster than GetSystemTimeAsFileTime(), a
    call takes a few ten nanoseconds."

    The above claim was corrected back in 2014. GetSystemTimePreciseAsFileTime() is considerably more expensive than GetSystemTimeAsFileTime(). It costs about 10 times more on a system that supports an invariant TSC. It would be more expensive on an older system that use the chipset's HPET or ACPI PM timer. That said, I doubt this matters very much when calling time.time() in interpreted code. It's a difference measured in tens of nanoseconds.

    @vstinner
    Copy link
    Member

    I wrote PR gh-116822 to use GetSystemTimePreciseAsFileTime() in time.time().

    By the way, I also modified modified time.monotonic() to use QueryPerformanceCounter(): commit 846ad5a of issue gh-88494.

    @vstinner
    Copy link
    Member

    vstinner commented Mar 18, 2024

    Feature implemented in 1d95451. For the delay @pitrou, it took 11 years to implement the function :-) I suppose that the delay was until all Windows versions supported by Python provide the function.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes OS-windows stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    Archived in project
    Development

    No branches or pull requests

    4 participants